Research Papers: Team Dynamics in D3

Concept Clustering in Design Teams: A Comparison of Human and Machine Clustering

[+] Author and Article Information
Chengwei Zhang

Department of Mechanical Engineering,
Tsinghua University,
A1003-1 Lizhaoji,
Beijing 100084, China;
Department of Mechanical Engineering,
University of California, Berkeley,
Berkeley, CA 94720
e-mail: zhangcw13@mails.tsinghua.edu.cn

Youngwook Paul Kwon

Department of Mechanical Engineering,
University of California, Berkeley,
2114 Etcheverry,
Berkeley, CA 94720
e-mail: young@berkeley.edu

Julia Kramer

Department of Mechanical Engineering,
University of California, Berkeley,
354/360 Hearst Memorial Mining Building,
Berkeley, CA 94720
e-mail: j.kramer@berkeley.edu

Euiyoung Kim

Jacobs Institute for Design Innovation,
University of California, Berkeley,
2530 Ridge Road,
Berkeley, CA 94720
e-mail: euiyoungkim@berkeley.edu

Alice M. Agogino

Fellow ASME
Department of Mechanical Engineering,
University of California, Berkeley,
415 Sutardja Dai Hall,
Berkeley, CA 94720
e-mail: agogino@berkeley.edu

1Corresponding author.

Contributed by the Design Theory and Methodology Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received February 20, 2017; final manuscript received July 18, 2017; published online October 2, 2017. Assoc. Editor: Harrison M. Kim.

J. Mech. Des 139(11), 111414 (Oct 02, 2017) (9 pages) Paper No: MD-17-1159; doi: 10.1115/1.4037478 History: Received February 20, 2017; Revised July 18, 2017

Concept clustering is an important element of the product development process. The process of reviewing multiple concepts provides a means of communicating concepts developed by individual team members and by the team as a whole. Clustering, however, can also require arduous iterations and the resulting clusters may not always be useful to the team. In this paper, we present a machine learning approach on natural language descriptions of concepts that enables an automatic means of clustering. Using data from over 1000 concepts generated by student teams in a graduate new product development class, we provide a comparison between the concept clustering performed manually by the student teams and the work automated by a machine learning algorithm. The goal of our machine learning tool is to support design teams in identifying possible areas of “over-clustering” and/or “under-clustering” in order to enhance divergent concept generation processes.

Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.


Tan, P. N. , Steinbach, M. , and Kumar, V. , 2005, “ Data Mining Cluster Analysis: Basic Concepts and Algorithms,” Introduction to Data Mining, Addison-Wesley, Boston, MA.
Ulrich, K. T. , and Eppinger, S. D. , 2016, Product Design and Development, 6th ed., McGraw-Hill, New York.
Goldstone, R. L. , and Kersten, A. , 2003, “ Concepts and Categorization,” Handbook of Psychology, Wiley, Hoboken, NJ. [CrossRef]
Smith, E. E. , 1989, “ Concepts and Induction,” Foundations of Cognitive Science, M. I. Posner , ed., MIT Press, Cambridge, MA, pp. 501–526.
Dong, A. , and Agogino, A. M. , 1996, “ Text Analysis for Constructing Design Representations,” Artificial Intelligence in Design, Springer, Dordrecht, The Netherlands, pp. 21–38. [PubMed] [PubMed]
Wood, W. H. , Yang, M. C. , Cutkosky, M. R. , and Agogino, A. M. , 2014, “ Design Information Retrieval: Improving Access to the Informal Side of Design,” ASME J. Mech. Des., 136(10), p. 101102.
Salonen, M. , and Perttula, M. , 2005, “ Utilization of Concept Selection Methods: A Survey of Finnish Industry,” ASME Paper No. DETC2005-85047.
Pugh, S. , 1996, Creative Innovative Products Using Total Design, Addison-Wesley, Boston, MA, p. 544.
Pugh, S. , 1981, “ Concept Selection: A Method That Works,” International Conference on Engineering Design, Rome, Italy, Mar. 9–13, pp. 497–506.
Roschuni, C. , Goodman, E. , and Agogino, A. M. , 2013, “ Communicating Actionable User Research for Human-Centered Design,” Artif. Intell. Eng. Des. Anal. Manuf., 27(2), pp. 143–154. [CrossRef]
Fisher, D. H. , 1987, “ Knowledge Acquisition Via Incremental Conceptual Clustering,” Mach. Learn., 2(2), pp. 139–172.
Lloyd, S. , 1982, “ Least Squares Quantization in PCM,” IEEE Trans. Inf. Theory, 28(2), pp. 129–137. [CrossRef]
Shi, J. , and Malik, J. , 2000, “ Normalized Cuts and Image Segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., 22(8), pp. 888–905. [CrossRef]
Comaniciu, D. , and Meer, P. , 2002, “ Mean Shift: A Robust Approach Toward Feature Space Analysis,” IEEE Trans. Pattern Anal. Mach. Intell., 24(5), pp. 603–619. [CrossRef]
Rokach, L. , and Maimon, O. , 2005, “ Clustering Methods,” Data Mining and Knowledge Discovery Handbook, Springer, New York, pp. 321–352. [CrossRef]
Bengio, Y. , Ducharme, R. , Vincent, P. , and Jauvin, C. , 2003, “ A Neural Probabilistic Language Model,” J. Mach. Learn. Res., 3, pp. 1137–1155.
Mikolov, T. , Chen, K. , Corrado, G. , and Dean, J. , 2013, “ Efficient Estimation of Word Representations in Vector Space,” preprint arXiv:1301.3781.
Levy, O. , and Goldberg, Y. , 2014, “ Dependency-Based Word Embeddings,” 52nd Annual Meeting of the Association for Computational Linguistics (ACL), Baltimore, MD, June 23–25, pp. 302–308.
Baroni, M. , Dinu, G. , and Kruszewski, G. , 2014, “ Don't Count, Predict! A Systematic Comparison of Context-Counting vs. Context-Predicting Semantic Vectors,” 52nd Annual Meeting of the Association for Computational Linguistics (ACL), Baltimore, MD, June 23–25, pp. 238–247.
LeCun, Y. , Bengio, Y. , and Hinton, G. , 2015, “ Deep Learning,” Nature, 521(7553), pp. 436–444. [CrossRef] [PubMed]
Xu, W. , Liu, X. , and Gong, Y. , 2003, “ Document Clustering Based on Non-Negative Matrix Factorization,” 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Toronto, ON, Canada, July 28–Aug. 1, pp. 267–273.
Aggarwal, C. C. , and Zhai, C. , 2012, “ A Survey of Text Clustering Algorithms,” Mining Text Data, Springer, New York, pp. 77–128. [CrossRef]
Steinbach, M. , Karypis, G. , and Kumar, V. , 2000, “ A Comparison of Document Clustering Techniques,” ACM Knowledge Discovery and Data Mining (KDD) Workshop on Text Mining, Boston, MA, Aug. 20–23, pp. 1–2.
Zhao, Y. , and Karypis, G. , 2005, “ Hierarchical Clustering Algorithms for Document Datasets,” Data Min. Knowl. Discovery, 10(2), pp. 141–168. [CrossRef]
Le, Q. , and Mikolov, T. , 2014, “ Distributed Representations of Sentences and Documents,” 31st International Conference on Machine Learning (ICML), Beijing, China, June 21–26, pp. 1188–1196.
Han, E. H. S. , and Karypis, G. , 2000, “ Centroid-Based Document Classification: Analysis and Experimental Results,” European Conference on Principles of Data Mining and Knowledge Discovery, Lyon, France, Sept. 13–16, pp. 424–431.
Manevitz, L. M. , and Yousef, M. , 2001, “ One-Class SVMs for Document Classification,” J. Mach. Learn. Res., 2, pp. 139–154.
Fuge, M. , Peters, B. , and Agogino, A. , 2014, “ Machine Learning Algorithms for Recommending Design Methods,” ASME J. Mech. Des., 136(10), p. 101103. [CrossRef]
Banerjee, S. , Ramanathan, K. , and Gupta, A. , 2007, “ Clustering Short Texts Using Wikipedia,” 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, The Netherlands, July 23–27, pp. 787–788.
Kuhn, A. , Ducasse, S. , and Gírba, T. , 2007, “ Semantic Clustering: Identifying Topics in Source Code,” Inf. Software Technol., 49(3), pp. 230–243. [CrossRef]
Fu, K. , Chan, J. , Cagan, J. , Kotovsky, K. , Schunn, C. , and Wood, K. , 2013, “ The Meaning of ‘Near’ and ‘Far’: The Impact of Structuring Design Databases and the Effect of Distance of Analogy on Design Output,” ASME J. Mech. Des., 135(2), p. 021007. [CrossRef]
Fu, K. , Cagan, J. , and Kotovsky, K. , 2010, “ Design Team Convergence: The Influence of Example Solution Quality,” ASME J. Mech. Des., 132(11), p. 111005. [CrossRef]
Gorla, A. , Tavecchia, I. , Gross, F. , and Zeller, A. , 2014, “ Checking App Behavior Against App Descriptions,” 36th ACM International Conference on Software Engineering (ICSE), Hyderabad, India, May 31–June 7, pp. 1025–1035.
Maalej, W. , and Nabil, H. , 2015, “ Bug Report, Feature Request, or Simply Praise? On Automatically Classifying App Reviews,” IEEE International on Requirements Engineering Conference (RE), Ottawa, ON, Canada, Aug. 24–28, pp. 116–125.
Sriram, B. , Fuhry, D. , Demir, E. , Ferhatosmanoglu, H. , and Demirbas, M. , 2010, “ Short Text Classification in Twitter to Improve Information Filtering,” 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Geneva, Switzerland, July 19–23, pp. 841–842.
Dong, A. , and Agogino, A. , 1997, “ Text Analysis for Constructing Design Representations,” Artif. Intell. Eng., 11(2), pp. 65–75. [CrossRef]
Pennington, J. , Socher, R. , and Manning, C. D. , 2014, “ Glove: Global Vectors for Word Representation,” Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, Oct. 25–29, pp. 1532–1543.
Wilkinson, L. , and Friendly, M. , 2009, “ The History of the Cluster Heat Map,” Am. Stat., 63(2), pp. 179–184. [CrossRef]


Grahic Jump Location
Fig. 1

Processing flow of our proposed method

Grahic Jump Location
Fig. 2

An example half-sheet from team 1

Grahic Jump Location
Fig. 3

Heat map of concepts generated by team 1

Grahic Jump Location
Fig. 4

A depiction of an HM plot. The x-axis shows the index of human clustering, the y-axis shows the index of machine clustering, and the size of the circles shows the number of concepts in each cluster.

Grahic Jump Location
Fig. 5

A depiction of the under- and over-clustering pattern of part of the clusters on an HM plot

Grahic Jump Location
Fig. 6

Overview of 11 teams' HM plots. In the subplot, the x-axis shows the human clusters, the y-axis shows the machine clusters, and the size of the circles shows the number of concepts in each cluster. More description of the HM plots are available under the “Supplemental Materials” tab for this paper on the ASME Digital Collection.

Grahic Jump Location
Fig. 8

Overview of under-clustering shown in teams' HM plots. Example: team 4 generated 12 concepts in cluster “children friendly designs” which the algorithm broke into six smaller clusters.

Grahic Jump Location
Fig. 7

Sample HM plot of team 1

Grahic Jump Location
Fig. 9

Overview of over-clustering shown in teams' HM plots. Example: team 3 created two clusters “outside exercise” and “cycling” but the algorithm combined these into one larger cluster.



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In