Research Papers: Design Theory and Methodology

Comparing the Effects of Design Interventions on the Quality of Design Concepts as a Reflection of Ideation Flexibility

[+] Author and Article Information
Daniel Henderson

Department of Engineering Science and
The Pennsylvania State University,
University Park, PA 16802

Kathryn Jablokow

School of Graduate Professional Studies,
The Pennsylvania State University,
Malvern, PA 19355

Shanna Daly

Department of Mechanical Engineering,
University of Michigan,
Ann Arbor, MI 48109

Seda McKilligan

Department of Industrial Design,
Iowa State University,
Ames, IA 50011

Eli Silk

Graduate School of Education,
Rutgers University,
New Brunswick, NJ 08901

Jennifer Bracken

Department of Mechanical and
Nuclear Engineering,
The Pennsylvania State University,
University Park, PA 16802

Contributed by the Design Theory and Methodology Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received July 16, 2018; final manuscript received November 13, 2018; published online January 10, 2019. Assoc. Editor: Katja Holtta-Otto.

J. Mech. Des 141(3), 031103 (Jan 10, 2019) (11 pages) Paper No: MD-18-1571; doi: 10.1115/1.4042048 History: Received July 16, 2018; Revised November 13, 2018

Many tools, techniques, and other interventions have been developed to support idea generation within the design process. In previous research, we explored the separate effects of three such design interventions: teaming, problem framing, and design heuristics. In the teaming intervention, participants discussed a design prompt together but recorded their own ideas separately. In problem framing, multiple versions (framings) of each design prompt were used to elicit different solutions. In design heuristics, participants used specially designed cards to prompt new ways of thinking about the given design problem. In the current work, we compared the effects of these three interventions on students' design ideas with respect to one idea attribute in particular—quality. In total, 1088 design concepts were collected from 171 undergraduate students in engineering and industrial design from two universities. Individual cognitive style was also assessed using Kirton's Adaption–Innovation inventory (KAI). Six metrics taken from the design literature were used to assess the quality of each concept, namely: acceptability, applicability, clarity, effectiveness, implementability, and implicational explicitness. Paired t-tests and Pearson correlations were used to assess differences in quality between concepts generated with and without the three interventions; in addition, secondary effects were sought based on the cognitive styles and academic standings of the participants. Statistically significant differences were observed in design concept quality for the teaming and design heuristics interventions over the full sample and for some subgroups separated by cognitive style and academic standing. These results have implications for how educators teach design interventions and how students choose and apply interventions to affect the quality of their own design solutions.

Copyright © 2019 by ASME
Your Session has timed out. Please sign back in to continue.


Shah, J. J. , Kulkarni, S. V. , and Vargas-Hernandez, N. , 2000, “ Evaluation of Idea Generation Methods for Conceptual Design: Effectiveness Metrics and Design of Experiments,” ASME J Mech. Des., 122(4), pp. 377–384. [CrossRef]
Daly, S. R. , Christian, J. L. , Yilmaz, S. , Seifert, C. M. , and Gonzalez, R. , 2012, “ Assessing Design Heuristics for Idea Generation in an Introductory Engineering Course,” Int. J. Eng. Educ., 28(2), pp. 463–473. https://lib.dr.iastate.edu/industrialdesign_pubs/5/
Silk, E. M. , Daly, S. R. , Jablokow, K. W. , Yilmaz, S. , and Rosenberg, M. , 2014, “ The Design Problem Framework: Using Adaption-Innovation Theory to Construct Design Problem Statements,” ASEE Annual Conference, Indianapolis, IN, June 15–18, Paper No. 8781. https://lib.dr.iastate.edu/industrialdesign_conf/7/
Yilmaz, S. , Daly, S. R. , Jablokow, K. , Silk, E. M. , and Rosenberg, M. , 2014, “ Investigating Impacts on the Ideation Flexibility of Engineers,” ASEE Annual Conference, Indianapolis, IN, June 15–18, Paper No. 8808 https://www.researchgate.net/publication/286743303_Investigating_impact_on_the_ideation_flexibility_of_engineers.
Rechkemmer, A. , Makhlouf, M. , Wenger, J. M. , Silk, E. M. , Daly, S. R. , McKilligan, S. , and Jablokow, K. W. , 2017, “ Examining the Effect of a Paradigm-Relatedness Problem-Framing Tool on Idea Generation,” ASEE Annual Conference, Columbus, OH, June 25–28, Paper No. 18507.
Wright, S. , Silk, E. M. , Daly, S. R. , Jablokow, K. W. , Yilmaz, S. , and Teerlink, W. , 2015, “ Exploring the Effects of Problem Framing on Solution Shifts: A Case Analysis,” ASEE Annual Conference, Seattle, WA, June 14–17, Paper No. 11638 https://lib.dr.iastate.edu/industrialdesign_conf/11/.
Dean, D. L. , Hender, J. M. , Rodgers, T. L. , and Santanen, E. L. , 2006, “ Identifying Quality, Novel, and Creative Ideas: Constructs and Scales for Idea Evaluation,” J. Assoc. Inf. Syst., 7(10), pp. 649–699. https://aisel.aisnet.org/jais/vol7/iss10/30/
Maher, M. , and Poon, J. , 1996, “ Modeling Design Exploration as Co-Evolution,” Comput. - Aided Civ. Infrastruct. Eng., 11(3), pp. 195–209. [CrossRef]
Cross, N. , 2004, “ Expertise in Design: An Overview,” Des. Stud., 25(5), pp. 427–441. [CrossRef]
Dorst, K. , and Cross, N. , 2001, “ Creativity in the Design Process: Co-Evolution of Problem-Solution,” Des. Stud., 22(5), pp. 425–437. [CrossRef]
Liu, Y. , Bligh, T. , and Chakrabarti, A. , 2003, “ Towards an 'Ideal' Approach for Concept Generation,” Des. Stud., 24(4), pp. 341–355. [CrossRef]
Yilmaz, S. , Seifert, C. , Daly, S. R. , and Gonzalez, R. , 2016, “ Design Heuristics in Innovative Products,” ASME J. Mech. Des., 138(7), p. 071102. [CrossRef]
Purcell, A. T. , and Gero, J. S. , 1996, “ Design and Other Types of Fixation,” Des. Stud., 17(4), pp. 363–383. [CrossRef]
Viswanathan, V. K. , and Linsey, J. S. , 2013, “ Design Fixation and Its Mitigation: A Study on the Role of Expertise,” ASME J. Mech. Des., 135(5), p. 051008. [CrossRef]
Jablokow, K. , and Kirton, M. J. , 2009, “ Problem Solving, Creativity, and the Level-Style Distinction,” Perspectives on the Nature of Intellectual Styles, L.-F. Zhang and R. J. Sternberg , eds., Springer, New York, pp. 137–168.
Kirton, M. J. , 2011, Adaption-Innovation in the Context of Diversity and Change, Routledge, London.
Sternberg, R. J. , 1988, The Triarchic Mind: A New Theory of Intelligence, Viking Press, New York.
Sternberg, R. J. , 1997, Thinking Styles, Cambridge University Press, New York.
Sternberg, R. J. , 2003, Cognitive Psychology, 3rd ed., Thomson Wadsworth, Belmont, CA.
Sternberg, R. J. , and Grigorenko, E. , 1997, “ Are Cognitive Styles Still in Style?,” Am. Psychol., 52(7), pp. 700–712. [CrossRef]
Sternberg, R. J. , and Grigorenko, E. L. , 2001, “ A Capsule History of Theory and Research on Styles.,” Perspectives on Thinking, Learning, and Cognitive Styles, R. J. Sternberg and L.-F. Zhang , eds., Lawrence Erlbaum Associates, Mahwah, NJ, pp. 1–22.
Abernathy, W. J. , and Utterback, J. M. , 1978, “ Patterns of Industrial Innovation,” Technol. Rev., 80(7), pp. 40–47. http://www.academia.edu/23341473/Patterns_of_Industrial_Innovation
Jablokow, K. W. , 2008, “ Developing Problem Solving Leadership: A Cognitive Approach,” Int. J. Eng. Educ., 24(5), pp. 936–954. https://www.researchgate.net/publication/233691102_Developing_Problem_Solving_Leadership_A_Cognitive_Approach
Samuel, P. , and Jablokow, K. W. , 2011, “ Toward an Adaption-Innovation Strategy for Engineering Design,” International Conference on Engineering Design (ICED), Copenhagen, Denmark, Aug. 15–18, pp. 377–386. https://www.designsociety.org/download-publication/30488/TOWARD+AN+ADAPTION-INNOVATION+STRATEGY+FOR+ENGINEERING+DESIGN
Valle, S. , and Vasquez-Bustelo, D. , 2009, “ Concurrent Engineering Performance: Incremental Versus Radical Innovation,” Int. J. Prod. Econ., 119(1), pp. 136–148. [CrossRef]
Carberry, A. R. , Lee, H.-S. , and Ohland, M. W. , 2010, “ Measuring Engineering Design Self-Efficacy,” J. Eng. Educ., 99(1), pp. 71–79. [CrossRef]
Felder, R. M. , and Brent, R. , 2005, “ Understanding Student Differences,” J. Eng. Educ., 94(1), pp. 57–72. [CrossRef]
Felder, R. M. , Felder, G. N. , and Dietz, E. J. , 2002, “ The Effects of Personality Type on Engineering Student Performance and Attitudes,” J. Eng. Educ., 91(1), pp. 3–17. [CrossRef]
Kirton, M. J. , and de Ciantis, S. M. , 1986, “ Cognitive Style and Personality: The Kirton Adaption–Innovation and Cattell's Sixteen Personality Factor Inventories,” Pers. Individ. Differ., 7(2), pp. 14–146. [CrossRef]
Goldsmith, R. E. , 1987, “ Creative Level and Creative Style,” Br. J. Soc. Psychol., 26(4), pp. 317–323. [CrossRef]
Hammerschmidt, P. K. , and Jennings, A. C. , 1992, “ The Impact of Personality Characteristics on Leadership Effectiveness Ratings,” Impact of Leadership, K. E. Clark , M. E. Clark , and D. P. Campbell , eds. Center for Creative Leadership, Greensboro, NC, pp. 469–475.
Buttner, E. H. , Gryskiewicz, N. , and Hidore, S. , 1999, “ The Relationship Between Styles of Creativity and Management Skills Assessment,” Br. J. Manage., 10(3), pp. 228–238. [CrossRef]
Silk, E. M. , Daly, S. R. , Jablokow, K. W. , Yilmaz, S. , Rechkemmer, A. , and Wenger, J. M. , 2016, “ Using Paradigm-Relatedness to Measure Design Ideation Shifts,” ASEE Annual Conference, New Orleans, LA, June 26–29, Paper No. 17004 https://www.asee.org/public/conferences/64/papers/17004/view.
Jablokow, K. W. , Teerlink, W. , Yilmaz, S. , Daly, S. , and Silk, E. , 2015, “ The Impact of Teaming and Cognitive Style on Student Perceptions of Design Ideation Outcomes,” ASEE Annual Conference, Seattle, WA, June 14–17, Paper No. 11855 https://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=1011&context=industrialdesign_conf.
Jablokow, K. , Teerlink, W. , Yilmaz, S. , Daly, S. , Silk, E. , and Wehr, C. , 2015, “ Ideation Variety in Mechanical Design: Examining the Effects of Cognitive Style and Design Heuristics,” ASME Paper No. DETC2015-46334.
DeFranco, J. F. , Jablokow, K. W. , Bilén, S. G. , and Gordon, A. , 2012, “ The Impact of Cognitive Style on Concept Mapping: Visualizing Variations in the Structure of Ideas,” ASEE Annual Conference, San Antonio, TX, June 10–13, Paper No. 3118.
Austin-Breneman, J. , Honda, T. , and Yang, M. C. , 2012, “ A Study of Student Design Team Behaviors in Complex System Design,” ASME J. Mech. Des., 134(12), p. 124504. [CrossRef]
Dinar, M. , Shah, J. , Cagan, J. , Leifer, L. J. , Linsey, J. S. , Smith, S. M. , and Vargas-Hernandez, N. , 2015, “ Empirical Studies of Designer Thinking: Past, Present, and Future,” ASME J. Mech. Des., 137(2), p. 021101. [CrossRef]
Fu, K. , Cagan, J. , and Kotovsky, K. , 2010, “ Design Team Convergence: The Influence of Example Solution Quality,” ASME J. Mech. Des., 132(11), p. 111005. [CrossRef]
Sosa, M. E. , and Marle, F. , 2013, “ Assembling Creative Teams in New Product Development Using Creative Team Familiarity,” ASME J. Mech. Des., 135(8), p. 081009. [CrossRef]
Page, S. , 2007, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies, Princeton University Press, Princeton, NJ.
Paulus, P. B. , and Yang, H. C. , 2000, “ Idea Generation in Groups: A Basis for Creativity in Organizations,” Organ. Behav. Human Decis. Processes, 82(1), pp. 76–87. [CrossRef]
Shergadwala, M. , Bilionis, I. , Kannan, K. , and Panchal, J. H. , 2018, “ Quantifying the Impact of Domain Knowledge and Problem Framing on Sequential Decisions in Engineering Design,” ASME J. Mech. Des., 140(10), p. 101402. [CrossRef]
Yilmaz, S. , Rosenberg, M. N. , Daly, S. R. , Jablokow, K. W. , Silk, E. M. , and Teerlink, W. , 2015, “ Impact of Problem Contexts on the Diversity of Design Solutions: An Exploratory Case Study,” ASEE Annual Conference, Seattle, WA, June 14–17, Paper No. 11206 https://www.researchgate.net/publication/285057385_Impact_of_problem_contexts_on_the_diversity_of_design_solutions_An_exploratory_case_study?ev=prf_high.
Daly, S. R. , Seifert, C. M. , Yilmaz, S. , and Gonzalez, R. , 2016, “ Comparing Ideation Techniques for Beginning Designers,” ASME J. Mech. Des., 138(10), p. 101108. [CrossRef]
Linsey, J. S. , Clauss, E. F. , Kurtoglu, T. , Murphy, J. T. , Wood, K. L. , and Markman, A. B. , 2011, “ An Experimental Study of Group Idea Generation Techniques: Understanding the Roles of Idea Representation and Viewing Methods,” ASME J. Mech. Des., 133(3), p. 031008. [CrossRef]
Zheng, X. , Ritter, S. C. , and Miller, S. R. , 2018, “ How Concept Selection Tools Impact the Development of Creative Ideas in Engineering Design Education,” ASME J. Mech. Des., 140(5), p. 052002. [CrossRef]
Daly, S. R. , Yilmaz, S. , Christian, J. L. , Seifert, C. M. , and Gonzalez, R. , 2012, “ Design Heuristics in Engineering Concept Generation,” J. Eng. Educ., 101(4), pp. 601–629. [CrossRef]
Yilmaz, S. , Daly, S. R. , Christian, J. L. , Seifert, C. M. , and Gonzalez, R. , 2014, “ Design Heuristics: Strategies to Inspire Ideas,” Design Heuristics, Des Moines, IA, accessed Nov. 17, 2017, www.designheuristics.com
Yilmaz, S. , Daly, S. R. , and Seifert, C. M. , 2014, “ Design Heuristics: An Evidence-Based Tool to Improve Innovation,” ASEE Annual Conference, Indianapolis, IN, June 15–18, Paper No. 8959. https://lib.dr.iastate.edu/cgi/viewcontent.cgi?referer=https://www.google.co.in/&httpsredir=1&article=1003&context=industrialdesign_conf
Shah, J. J. , Smith, S. M. , and Vargas-Hernandez, N. , 2003, “ Metrics for Measuring Ideation Effectiveness,” Des. Stud., 24(2), pp. 111–134. [CrossRef]
Besemer, S. , 1998, “ Creative Product Analysis Matrix: Testing the Model Structure and a Comparison Among Products-Three Novel Chairs,” Creativity Res. J., 11(4), pp. 333–346. [CrossRef]
Ahmed, F. , and Fuge, M. , 2017, “ Ranking Ideas for Diversity and Quality,” ASME J. Mech. Des., 140(1), p. 011101. [CrossRef]
Henderson, D. , Helm, K. , Jablokow, K. W. , McKilligan, S. , Daly, S. R. , and Silk, E. M. , 2017, “ A Comparison of Variety Metrics in Engineering Design,” ASME Paper No. DETC2017-67502.
Helm, K. C. , Jablokow, K. W. , Daly, S. R. , Silk, E. M. , Yilmaz, S. , and Suero, R. , 2016, “ Evaluating the Impacts of Different Interventions on Quality in Concept Generation,” ASEE Annual Conference, New Orleans, LA, June 26–29, Paper No. 15648 https://www.asee.org/public/conferences/64/papers/15648/view.
Edelman, J. A. , 2011, “ Understanding Radical Breaks: Media and Behavior in Small Teams Engaged in Redesign Scenarios,” Ph.D. thesis, Stanford University, Stanford, CA.
Sarkar, P. , and Chakrabarti, A. , 2011, “ Assessing Design Creativity,” Des. Stud., 32(4), pp. 348–383. [CrossRef]
Yilmaz, S. , Daly, S. R. , Christian, J. L. , Seifert, C. M. , and Gonzalez, R. , 2014, “ Can Experienced Designers Learn From New Tools? A Case Study of Idea Generation in a Professional Engineering Team,” Int. J. Des. Creativity Innovation, 2(2), pp. 82–96. [CrossRef]
Sonalkar, N. , Mabogunje, A. , and Cutkosky, M. , 2018, “ Quadratic Model of Reciprocal Causation for Monitoring, Improving, and Reflecting on Design Team Performance,” Design Thinking Research, Understanding Innovation, H. Plattner ed., Springer, New York, pp. 43–57.
Buffington, K. W. , Jablokow, K. W. , and Martin, K. A. , 2002, “ Project Team Dynamics and Cognitive Style,” Eng. Manage. J., 14, pp. 25–33. [CrossRef]
DeFranco, J. , and Neill, C. , 2011, “ Problem-Solving Style and Its Impact on Engineering Team Effectiveness,” Annual Conference on Systems Engineering Research, Los Angeles, CA, Apr. 14–16, pp. 1–13.
Vercellone-Smith, P. , Jablokow, K. W. , and Friedel, C. , 2012, “ Characterizing Communication Networks in a Web-Based Classrom: Cognitive Styles and Linguistic Behavior of Self-Organizing Groups in Online Discussions,” Comput. Educ., 59, pp. 222–235. [CrossRef]
Baer, J. , 2014, Creativity and Divergent Thinking: A Task-Specific Approach, Psychology Press, London.
Niku, S. B. , 2008, Creative Design of Products and Systems, Wiley, Hoboken, NJ.
Teerlink, W. , 2015, “ Team Ideation: A Detailed Investigation Into Ideation Effectiveness,” Master's thesis, The Pennsylvania State University, University Park, PA. https://etda.libraries.psu.edu/catalog/26388
Laughlin, P. , 2002, “ Groups Perform Better Than the Best Individuals on Letters-to-Numbers Problems,” Organ. Behav. Human Decis. Processes, 88(2), pp. 605–620. [CrossRef]
Linsey, J. S. , Murphy, J. T. , and Markman, A. B. , 2006, “ Representing Analogies: Increasing the Probability of Innovation,” ASME Paper No. DETC2006-99383.
Diehl, M. , and Stroebe, W. , 1987, “ Productivity Loss in Brainstorming Groups: Toward the Solution of a Riddle,” J. Pers. Soc. Psychol., 53(3), pp. 497–509. [CrossRef]
Mullen, B. , Johnson, C. , and Salas, E. , 1991, “ Productivity Loss in Brainstorming Groups: A Meta-Analytic Integration,” Basic Appl. Soc. Psychol., 12(1), pp. 3–23. [CrossRef]
Choo, P. K. , Lou, Z. N. , Camburn, B. A. , Wood, K. L. , Koo, B. , and Grey, F. , 2014, “ Ideation Methods: A First Study on Measured Outcomes With Personality Type,” ASME Paper No. DETC2014-34954.
Berthelsdorf, F. A. , and Stone, R. B. , 2017, “ Creativity of Concept Ideation as Affected by Team Personality,” ASME Paper No. DETC2017-67974.
Jablokow, K. W. , and Booth, D. E. , 2006, “ The Impact and Management of Cognitive Gap in High Performance Product Development Organizations,” J. Eng. Technol. Manage., 23(4), pp. 313–336. [CrossRef]


Grahic Jump Location
Fig. 2

DPF: comparison of adaptive and innovative framings. Adapted from [3].

Grahic Jump Location
Fig. 1

Visual summary of the ideation flexibility project. Adapted from [4].

Grahic Jump Location
Fig. 3

An example of the 77 DH cards ideation tool (front and back of card #76) [2] (Reprinted with permission of Design Heuristics LLC, copyright 2009)

Grahic Jump Location
Fig. 4

Example flow of an ideation flexibility data collection session [4]



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In