Research Papers: Design Education

How Concept Selection Tools Impact the Development of Creative Ideas in Engineering Design Education

[+] Author and Article Information
Xuan Zheng

Department of Industrial
and Manufacturing Engineering,
The Pennsylvania State University,
343 Leonhard Building,
University Park, PA 16802
e-mail: xxz159@psu.edu

Sarah C. Ritter

School of Engineering Design,
Technology and Professional Programs,
The Pennsylvania State University,
213 Hammond Building,
University Park, PA 16802-1401
e-mail: scr15@psu.edu

Scarlett R. Miller

School of Engineering Design,
Technology and Professional Programs,
The Pennsylvania State University,
213-P Hammond Building,
University Park, PA 16802-1401
e-mail: scarlettmiller@psu.edu

1Corresponding author.

Contributed by the Design Education Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received June 12, 2017; final manuscript received January 17, 2018; published online March 23, 2018. Assoc. Editor: Irem Tumer.

J. Mech. Des 140(5), 052002 (Mar 23, 2018) (11 pages) Paper No: MD-17-1401; doi: 10.1115/1.4039338 History: Received June 12, 2017; Revised January 17, 2018

Concept selection tools have been heavily integrated into engineering design education in an effort to reduce the risks and uncertainties of early-phase design ideas and aid students in the decision-making process. However, little research has examined the utility of these tools in promoting creative ideas or their impact on student team decision making throughout the conceptual design process. To fill this research gap, the current study was designed to compare the impact of two concept selection tools, the concept selection matrix (CSM) and the tool for assessing semantic creativity (TASC) on the average quality (AQL) and average novelty (ANV) of ideas selected by student teams at several decision points throughout an 8-week project. The results of the study showed that the AQL increased significantly in the detailed design stage, while the ANV did not change. However, this change in idea quality was not significantly impacted by the concept selection tool used, suggesting other factors may impact student decision making and the development of creative ideas. Finally, student teams were found to select ideas ranked highly in concept selection tools only when these ideas met their expectations, indicating that cognitive biases may be significantly impeding decision making.

Copyright © 2018 by ASME
Your Session has timed out. Please sign back in to continue.


Mattson, C. A. , and Messac, A. , 2005, “Pareto Frontier Based Concept Selection Under Uncertainty, With Visualization,” Optim. Eng., 6(1), pp. 85–115. [CrossRef]
Pahl, G. , Beitz, W. , Feldhusen, J. , and Grote, K. , 2007, Engineering Design: A Systematic Approach, 3rd ed., Springer Science+Business Media Deutschland GmbH, Berlin, p. 632.
Nikander, J. B. , Liikkanen, L. A. , and Laakso, M. , 2014, “The Preference Effect in Design Concept Evaluation,” Des. Stud., 35(5), pp. 473–499. [CrossRef]
Ulrich, K. T. , Eppinger, S. D. , and Goyal, A. , 2011, Product Design and Development, McGraw-Hill, New York.
Huang, H.-Z. , Liu, Y. , Li, Y. , Xue, L. , and Wang, Z. , 2013, “New Evaluation Methods for Conceptual Design Selection Using Computational Intelligence Techniques,” J. Mech. Sci. Technol., 27(3), pp. 733–746. [CrossRef]
Amabile, T. M. , 1983, “The Social Psychology of Creativity: A Componential Conceptualization,” J. Pers. Soc. Psychol., 45(2), pp. 357–376. [CrossRef]
Brown, R. T. , 1989, “Creativity,” Handbook of Creativity, Springer, Boston, MA, pp. 3–32. [PubMed] [PubMed]
Mayer, R. E. , 1999, “Fifty Years of Creativity Research,” Handbook of Creativity, The Press Syndicate of the University of Cambridge, Cambridge, UK, p. 449. [CrossRef]
Shah, J. J. , Vargas-Hernandez, N. , and Smith, S. M. , 2003, “Metrics for Measuring Ideation Effectiveness,” Des. Stud., 24(2), pp. 111–134. [CrossRef]
Goel, P. S. , and Singh, N. , 1998, “Creativity and Innovation in Durable Product Development,” Comput. Ind. Eng., 35(1–2), pp. 5–8. [CrossRef]
Dym, C. L. , Agogino, A. M. , Eris, O. , Frey, D. D. , and Leifer, L. J. , 2005, “Engineering Design Thinking, Teaching, and Learning,” J. Eng. Educ., 94(1), pp. 103–120. [CrossRef]
Daly, S. R. , Mosyjowski, E. A. , and Seifert, C. M. , 2014, “Teaching Creativity in Engineering Courses,” J. Eng. Educ., 103(3), pp. 417–449. [CrossRef]
Rietzschel, E. F. , Nijstad, B. A. , and Stroebe, W. , 2006, “Productivity is Not Enough: A Comparison of Interactive and Nominal Groups in Idea Generation and Selection,” J. Exp. Soc. Psychol., 42(2), pp. 244–251. [CrossRef]
Starkey, E. , Gosnell, C. A. , and Miller, S. R. , 2015, “Implementing Creativity Evaluation Tools Into the Concept Selection Process in Engineering Education,” ASME Paper No. DETC2015-47396.
Gosnell, C. A. , and Miller, S. R. , 2016, “But is it Creative? Delineating the Impact of Expertise and Concept Ratings on Creative Concept Selection,” ASME J. Mech. Des., 138(2), p. 021101. [CrossRef]
Toh, C. , and Miller, S. , 2014, “The Role of Individual Risk Attitudes on the Selection of Creative Concepts in Engineering Design,” ASME Paper No. DETC2014-35106.
Toh, C. , Patel, A. , Strohmetz, A. , and Miller, S. , 2015, “My Idea is Best! Ownership Bias and Its Influence in Engineering Concept Selection,” ASME Paper No. DETC2015-46478.
Toh, C. A. , and Miller, S. R. , 2015, “How Engineering Teams Select Design Concepts: A View Through the Lens of Creativity,” Des. Stud., 38, pp. 111–138. [CrossRef]
Starkey, E. , Toh, C. A. , and Miller, S. R. , 2016, “Abandoning Creativity: The Evolution of Creative Ideas in Engineering Design Course Projects,” Des. Stud., 47, pp. 47–72. [CrossRef]
Sarbacker, S. D. , and Ishii, K. , 1997, “A Framework for Evaluating Risk in Innovative Product Development,” Design Engineering Technical Conferences, Sacramento, CA, Sept. 14–17, pp. 1–10.
Ogot, M. , and Okudan-Kremer, G. E. , 2006, Engineering Design: A Practical Guide, Trafford Publishing, Bloomington, IN. [PubMed] [PubMed]
Pugh, S. , 1991, Total Design: Integrated Methods for Successful Product Engineering, Addison-Wesley, Workingham, UK.
Akao, Y. , 1994, “Development History of Quality Function Deployment,” The Customer Driven Approach to Quality Planning and Deployment, Asian Productivity Organization, Minato, Japan, p. 339.
Saaty, T. L. , 1994, “How to Make a Decision: The Analytic Hierarchy Process,” Interfaces, 24(6), pp. 19–43. [CrossRef]
Marsh, E. R. , Slocum, A. H. , and Otto, K. N. , 1993, Hierarchical Decision Making in Machine Design, MIT Precision Engineering Research Center, Cambridge, MA.
Racheva, Z. , Daneva, M. , and Buglione, L. , 2008, “Supporting the Dynamic Reprioritization of Requirements in Agile Development of Software Products,” Second International Workshop on Software Product Management (IWSPM'08), Barcelona, Spain, Sept. 9, pp. 49–58.
Hurst, K. , 1999, Engineering Design Principles, Butterworth-Heinemann, New York.
Nickerson, R. S. , 1998, “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises,” Rev. Gen. Psychol., 2(2), p. 175. [CrossRef]
Martin, M. W. , 2006, “Moral Creativity in Science and Engineering,” Sci. Eng. Ethics, 12(3), pp. 421–433. [CrossRef] [PubMed]
Fischer, G. , 2013, “Learning, Social Creativity, and Cultures of Participation,” Learning and Collective Creativity: Activity-Theoretical and Sociocultural Studies, Routledge, New York, p. 198.
Harms, R. , and Van der Zee, K. , 2013, “Interview: Paul Paulus Group Creativity,” Creativity Innovation Manage., 22(1), pp. 96–99. [CrossRef]
Gosnell, C. A. , and Miller, S. R. , 2014, “A Novel Method for Providing Global Assessments of Design Concepts Using Single-Word Adjectives and Semantic Similarity,” ASME Paper No. DETC2014-35380.
Triantaphyllou, E. , and Mann, S. H. , 1995, “Using the Analytic Hierarchy Process for Decision Making in Engineering Applications: Some Challenges,” Int. J. Ind. Eng.: Appl. Pract., 2(1), pp. 35–44. https://www.researchgate.net/publication/241416054_Using_the_analytic_hierarchy_process_for_decision_making_in_engineering_applications_Some_challenges
Parnes, S. J. , and Meadow, A. , 1959, “Effects of ‘Brainstorming’ Instructions on Creative Problem Solving by Trained and Untrained Subjects,” J. Educ. Psychol., 50(4), p. 171. [CrossRef]
Dennis, A. R. , and Valacich, J. S. , 1993, “Computer Brainstorms: More Heads are Better Than One,” J. Appl. Psychol., 78(4), p. 531. [CrossRef]
Osborn, A. , 1957, Applied Imagination, Scribner, New York.
Shah, J. , 1993, “Method 5-1-4 G-a Variation on Method 635,” MAE 540 Class Notes, Arizona State University, Tempe, AZ.
Kazerounian, K. , and Foley, S. , 2007, “Barriers to Creativity in Engineering Education: A Study of Instructors and Students Perceptions,” ASME J. Mech. Des., 129(7), pp. 761–768. [CrossRef]
Rietzschel, E. , Nijstad, B. A. , and Stroebe, W. , 2010, “The Selection of Creative Ideas After Individual Idea Generation: Choosing Between Creativity and Impact,” Br. J. Psychol., 101(1), pp. 47–68. [CrossRef] [PubMed]
Wilde, D. J. , 1997, “Using Student Preferences to Guide Design Team Composition,” Design Engineering Technical Conferences (DETC), Sacramento, CA, Sept. 14–17, pp. 1–6. http://web.engr.oregonstate.edu/~paasch/classes/me382/3890.PDF
Toh, C. A. , and Miller, S. R. , 2016, “Creativity in Design Teams: The Influence of Personality Traits and Risk Attitudes on Creative Concept Selection,” Res. Eng. Des., 27(1), pp. 73–89. [CrossRef]
Toh, C. A. , and Miller, S. R. , 2016, “Choosing Creativity: The Role of Individual Risk and Ambiguity Aversion on Creative Concept Selection in Engineering Design,” Res. Eng. Des., 27(3), pp. 195–219. [CrossRef]
Jansson, D. , and Smith, S. , 1991, “Design Fixation,” Des. Stud., 12(1), pp. 3–11. [CrossRef]
Toh, C. A. , Miller, S. R. , and Kremer, G. E. , 2013, “The Role of Personality and Team-Based Product Dissection on Fixation Effects,” Adv. Eng. Educ., 3(4), pp. 1–23. https://files.eric.ed.gov/fulltext/EJ1076107.pdf
Toh, C. A. , Miller, S. R. , and Kremer, G. E. , 2012, “Mitigating Design Fixation Effects in Engineering Design Through Product Dissection Activities,” Design Computing and Cognition, College Station, TX, June 7–9, pp. 95–113.
Toh, C. , Strohmetz, A. , and Miller, S. , 2016, “The Effects of Gender and Idea Goodness on Ownership Bias in Engineering Design Education,” ASME J. Mech. Des., 138(10), p. 101105.
Mueller, J. S. , Melwani, S. , and Goncalo, J. A. , 2012, “The Bias Against Creativity: Why People Desire but Reject Creative Ideas,” Psychol. Sci., 23(1), pp. 13–17. [CrossRef] [PubMed]
Rubenson, D. L. , and Runco, M. A. , 1995, “The Psychoeconomic View of Creative Work in Groups and Organizations,” Creativity Innovation Manag., 4(4), pp. 232–241.
Runco, M. A. , and Charles, R. E. , 1993, “Judgements of Originality and Appropriateness as Predictors of Creativity,” Pers. Individ. Differ., 15(5), pp. 537–546. [CrossRef]
Linnerud, B. , and Mocko, G. , 2013, Factors That Effect Motivation and Performance on Innovative Design Projects,” ASME Paper No. DETC2013-12758.
Saaty, T. L. , 1990, “How to Make a Decision: The Analytic Hierarchy Process,” Eur. J. Oper. Res., 48(1), pp. 9–26. [CrossRef]
Ulrich, K. T. , and Eppinger, S. , 2007, Product Design and Development, McGraw-Hill, New York.
Saaty, T. L. , 1980, The Analytic Hierarchy Process, McGraw-Hill, New York.
Ogot, M. , and Okudan-Kremer, G. , 2006, Engineering Design: A Practical Guide, Trafford Publishing, St. Victoria, BC, Canada. [PubMed] [PubMed]
Lai, V. S. , Wong, B. K. , and Cheung, W. , 2002, “Group Decision Making in a Multiple Criteria Environment: A Case Using the AHP in Software Selection,” Eur. J. Oper. Res., 137(1), pp. 134–144. [CrossRef]
Korpela, J. , and Tuominen, M. , 1997, “Group Decision Support for Analysing Logistics Development Projects,” 30th Hawaii International Conference on System Sciences, Wailea, HI, Jan. 7–11, pp. 493–502.
Dyer, R. F. , and Forman, E. H. , 1992, “Group Decision Support With the Analytic Hierarchy Process,” Decis. Support Syst., 8(2), pp. 99–124. [CrossRef]
Tam, M. C. , and Tummala, V. R. , 2001, “An Application of the AHP in Vendor Selection of a Telecommunications System,” Omega, 29(2), pp. 171–182. [CrossRef]
Levine, J. M. , 1989, “Reaction to Opinion Deviance in Small Groups,” Psychology of Group Influence, 2nd ed., Psychology Press, New York, pp. 187–231.
Naquin, C. E. , and Tynan, R. O. , 2003, “The Team Halo Effect: Why Teams are Not Blamed for Their Failures,” J. Appl. Psychol., 88(2), p. 332. [CrossRef] [PubMed]
Larey, T. S. , and Paulus, P. B. , 1999, “Group Preference and Convergent Tendencies in Small Groups: A Content Analysis of Group Brainstorming Performance,” Creativity Res. J., 12(3), pp. 175–184. [CrossRef]
Salonen, M. , and Perttula, M. , 2005, “Utilization of Concept Selection Methods: A Survey of Finnish Industry,” ASME Paper No. DETC2005-85047.
Katsikopoulos, K. V. , 2012, “Decision Methods for Design: Insights From Psychology,” ASME J. Mech. Des., 134(8), p. 084504. [CrossRef]
Whitaker, R. , 2007, “Criticisms of the Analytic Hierarchy Process: Why They Often Make No Sense,” Math. Comput. Model., 46(7–8), pp. 948–961. [CrossRef]
Belton, V. , and Gear, T. , 1985, “The Legitimacy of Rank Reversal—A Comment,” Omega, 13(3), pp. 143–144. [CrossRef]
Dyer, J. S. , 1990, “Remarks on the Analytic Hierarchy Process,” Manage. Sci., 36(3), pp. 249–258. [CrossRef]
Pérez, J. , Jimeno, J. L. , and Mokotoff, E. , 2006, “Another Potential Shortcoming of AHP,” Top, 14(1), pp. 99–111. [CrossRef]
Surowiecki, J. , 2005, The Wisdom of Crowds, Anchor, New York.
Kerr, N. L. , and Tindale, R. S. , 2011, “Group-Based Forecasting?: A Social Psychological Analysis,” Int. J. Forecasting, 27(1), pp. 14–40. [CrossRef]
Amabile, T. , 1982, “Social Psychology of Creativity: A Consensusual Assessment Technique,” J. Pers. Soc. Psychol., 43(5), pp. 997–1013. [CrossRef]
Hallihan, G. M. , Cheong, H. , and Shu, L. , 2012, “Confirmation and Cognitive Bias in Design Cognition,” ASME Paper No. DETC2012-71258.
Gill, H. , 1990, “Adoption of Design Science by Industry—Why so Slow?,” J. Eng. Des., 1(3), pp. 289–295. [CrossRef]
Maurer, C. , and Widmann, J. , 2012, “Conceptual Design Theory in Education Versus Practice in Industry: A Comparison Between Germany and the United States,” ASME Paper No. DETC2012-70079.
Toh, C. , Miele, L. , and Miller, S. , 2015, “Which One Should I Pick? Concept Selection in Engineering Design Industry,” ASME Paper No. DETC2015-46522.
Jonas, E. , Schulz-Hardt, S. , Frey, D. , and Thelen, N. , 2001, “Confirmation Bias in Sequential Information Search After Preliminary Decisions: An Expansion of Dissonance Theoretical Research on Selective Exposure to Information,” J. Pers. Soc. Psychol., 80(4), p. 557. [CrossRef] [PubMed]
Ask, K. , and Granhag, P. A. , 2005, “Motivational Sources of Confirmation Bias in Criminal Investigations: The Need for Cognitive Closure,” J. Invest. Psychol. Offender Profiling, 2(1), pp. 43–63. [CrossRef]
Zheng, X. , and Miller, S. R. , 2016, “How Do I Choose? the Influence of Concept Selection Methods on Student Team Decision-Making,” ASME Paper No. DETC2016-60333.
Osborn, A. F. , Rona, G. , Dupont, P. , and Armand, L. , 1971, The Constructive Imagination: How to Take Advantage of Its Ideas, Principles and Process of the Creative Thought and Brainstorming, Dunod, Paris, France.
Linsey, J. S. , Clauss, E. F. , Kurtoglu, T. , Murphy, J. T. , Wood, K. L. , and Markman, A. B. , 2011, “An Experimental Study of Group Idea Generation Techniques: Understanding the Roles of Idea Representation and Viewing Methods,” ASME J. Mech. Des., 133(3), p. 031008. [CrossRef]
Carley, K. , 1990, “Content Analysis,” The Encyclopedia of Language and Linguistics, R. E. Asher , ed., Pergamon Press, Edinburgh, UK.
Genco, N. , Holtta-Otto, K. , and Seepersad, C. C. , 2012, “An Experimental Investigation of the Innovation Capabilities of Undergraduate Engineering Students,” J. Eng. Educ., 101(1), pp. 60–81. [CrossRef]


Grahic Jump Location
Fig. 1

Screenshot of the TASC website that asking participants to choose 3–5 adjectives from a word bank

Grahic Jump Location
Fig. 2

The timetable of the project and the in-class design practices

Grahic Jump Location
Fig. 3

Example of the idea ranked first by team 5 from the TASC section with a creativity score of 4.166

Grahic Jump Location
Fig. 4

Ideas that were averaged for the novelty and quality ateach stage (Note: rankings in concept selection tools were not necessarily the same with rankings in team informal discussion)

Grahic Jump Location
Fig. 6

The evolution of the AQL throughout the design process (data over the bars show the means of the variables; bars represent standard errors)

Grahic Jump Location
Fig. 5

The evolution of the ANV throughout the design process (data over the bars show the means of the variables; bars represent standard errors)

Grahic Jump Location
Fig. 7

Comparing ANV throughout the design process in TASC and CSM section (data over the bars show means of variables; bars represent for standard errors)

Grahic Jump Location
Fig. 8

Comparing AQL throughout the design process in TASC and CSM section (data over the bars show means of variables; bars represent for standard errors)

Grahic Jump Location
Fig. 9

Percentages of student teams that selected ideas with different rankings by concept selection tools



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In