Research Papers: Design Theory and Methodology

Machine Learning Algorithms for Recommending Design Methods

[+] Author and Article Information
Mark Fuge

Department of Mechanical Engineering,
Berkeley Institute of Design,
University of California,
Berkeley, CA 94709
e-mail: mark.fuge@berkeley.edu

Bud Peters

Department of Mathematics,
Berkeley Institute of Design,
University of California,
Berkeley, CA 94709
e-mail: dbpeters@berkeley.edu

Alice Agogino

Department of Mechanical Engineering,
Berkeley Institute of Design,
University of California,
Berkeley, CA 94709
e-mail: agogino@berkeley.edu

1Corresponding author.

Contributed by the Design Theory and Methodology Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received January 9, 2014; final manuscript received July 21, 2014; published online August 18, 2014. Assoc. Editor: Irem Y. Tumer.

J. Mech. Des 136(10), 101103 (Aug 18, 2014) (8 pages) Paper No: MD-14-1013; doi: 10.1115/1.4028102 History: Received January 09, 2014; Revised July 21, 2014

Every year design practitioners and researchers develop new methods for understanding users and solving problems. This increasingly large collection of methods causes a problem for novice designers: How does one choose which design methods to use for a given problem? Experienced designers can provide case studies that document which methods they used, but studying these cases to infer appropriate methods for a novel problem is inefficient. This research addresses that issue by applying techniques from content-based and collaborative filtering to automatically recommend design methods, given a particular problem. Specifically, we demonstrate the quality with which different algorithms recommend 39 design methods out of an 800+ case study dataset. We find that knowing which methods occur frequently together allows one to recommend design methods more effectively than just using the text of the problem description itself. Furthermore, we demonstrate that automatically grouping frequently co-occurring methods using spectral clustering replicates human-provided groupings to 92% accuracy. By leveraging existing case studies, recommendation algorithms can help novice designers efficiently navigate the increasing array of design methods, leading to more effective product design.

Copyright © 2014 by ASME
Your Session has timed out. Please sign back in to continue.


Roschuni, C., Agogino, A., and Beckman, S., 2011, “The DesignExchange: Supporting the Design Community of Practice,” International Conference on Engineering Design, International Conference on Engineering Design (ICED '11), Vol. 8, pp. 255–264.
Broadbent, G., and Ward, A., 1969, Design Methods in Architecture, AA Papers, Lund Humphries.
Broadbent, G., 1979, “The Development of Design Methods,” Des. Methods Theor., 13(1), pp. 41–45.
Jones, J. C., 1992, Design Methods, 2nd ed. Wiley, John Wiley and Sons, New York.
Margolin, V., and Buchanan, G. R., 1996, The Idea of Design, The MIT Press, Cambridge, MA.
McCarthy, J. M., 2005, “Engineering Design in 2030: Human Centered Design,” ASME J. Mech. Des., 127(3), p. 357. [CrossRef]
Collopy, P., 2013, “Opportunities in Engineering Design Research,” ASME J. Mech. Des., 135(2), p. 020301. [CrossRef]
Van Pelt, A., and Hey, J., 2011, “Using TRIZ and Human-Centered Design for Consumer Product Development,” Procedia Eng., 9, pp. 688–693. [CrossRef]
Altshuller, G., Shulyak, L., Rodman, S., and Fedoseev, U., 1998, 40 Principles: TRIZ Keys to Innovation, Vol. 1, Technical Innovation Center Inc., Worcester, MA.
Resnick, P., and Varian, H. R., 1997, “Recommender Systems,” Commun. ACM, 40(3), pp. 56–58. [CrossRef]
Adomavicius, G., and Tuzhilin, A., 2005, “Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions,” IEEE Trans. Knowledge Data Eng., 17(6), pp. 734–749. [CrossRef]
Manning, C. D., Raghavan, P., and Schütze, H., 2008, Introduction to Information Retrieval, Cambridge University Press, New York.
Page, L., Brin, S., Motwani, R., and Winograd, T., 1999, “The PageRank Citation Ranking: Bringing Order to the Web,” Technical Report No. 1999-66, Stanford InfoLab. Previous No. SIDL-WP-1999-0120.
Deerwester, S. C., Dumais, S. T., Landauer, T. K., Furnas, G. W., and Harshman, R. A., 1990, “Indexing by Latent Semantic Analysis,” JASIS, 41(6), pp. 391–407. [CrossRef]
Dong, A., Hill, A. W., and Agogino, A. M., 2004, “A Document Analysis Method for Characterizing Design Team Performance,” ASME J. Mech. Des., 126(3), pp. 378–385. [CrossRef]
Blei, D. M., Ng, A. Y., and Jordan, M. I., 2003, “Latent Dirichlet Allocation,” J. Mach. Learn. Res., 3, pp. 993–1022.
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., and Hullender, G., 2005, “Learning to Rank Using Gradient Descent,” Proceedings of the 22nd International Conference on Machine Learning, ICML’05, ACM, pp. 89–96. [CrossRef]
Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., and Li, H., 2007, “Learning to Rank: From Pairwise Approach to Listwise Approach,” Proceedings of the 24th International Conference on Machine Learning, ICML’07, ACM, pp. 129–136. [CrossRef]
Liu, T.-Y., 2007, “Learning to Rank for Information Retrieval,” Found Trends Inf. Retrieval, 3(3), pp. 225–331. [CrossRef]
Herlocker, J. L., Konstan, J. A., Borchers, A., and Riedl, J., 1999, “An Algorithmic Framework for Performing Collaborative Filtering,” Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’99, ACM, pp. 230–237. [CrossRef]
Bell, R. M., Koren, Y., and Volinsky, C., 2007, The BellKor Solution to the Netflix Prize.
Salakhutdinov, R., and Mnih, A., 2008, “Bayesian Probabilistic Matrix Factorization Using Markov Chain Monte Carlo,” Proceedings of the 25th International Conference on Machine Learning, ICML’08, ACM, pp. 880–887. [CrossRef]
Nazemian, A., Gholami, H., and Taghiyareh, F., 2012, “An Improved Model of Trust-Aware Recommender Systems Using Distrust Metric,” IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pp. 1079–1084. [CrossRef]
Badaro, G., Hajj, H., El-Hajj, W., and Nachman, L., 2013, “A Hybrid Approach With Collaborative Filtering for Recommender Systems,” 9th International Wireless Communications and Mobile Computing Conference (IWCMC), pp. 349–354. [CrossRef]
Ghazanfar, M. A., and Prugel-Bennett, A., 2010, “A Scalable, Accurate Hybrid Recommender System,” Proceedings of 3rd International Conference on Knowledge Discovery and Data Mining, pp. 94–98. [CrossRef]
Freund, Y., and Schapire, R. E., 1997, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,” J. Comput. Syst. Sci., 55(1), pp. 119–139. [CrossRef]
Yujie, Z., and Licai, W., 2010, “Some Challenges for Context-Aware Recommender Systems,” 5th International Conference on Computer Science and Education (ICCSE), pp. 362–365. [CrossRef]
Jones, J. C., and Thornley, D., eds., 1962, Conference on Design Methods: Papers Presented at the Conference on Systematic and Intuitive Methods in Engineering, Industrial Design, Architecture and Communications, Pergamon, Pergamon Press, Oxford, UK.
Helen Hamlyn Centre for Design, 2013, “Designing With People: Methods,” http://designingwithpeople.rca.ac.uk/methods
Panchal, J. H., and Messer, M., 2011, “Extracting the Structure of Design Information From Collaborative Tagging,” ASME J. Comput. Inf. Sci. Eng., 11(4), p. 041007. [CrossRef]
Li, Z., and Ramani, K., 2007, “Ontology-Based Design Information Extraction and Retrieval,” AI EDAM, 21(4), pp. 137–154. [CrossRef]
Suh, N. P., 2001, Axiomatic Design: Advances and Applications (The Oxford Series on Advanced Manufacturing), Oxford University, Oxford University Press, New York.
Pahl, G., Beitz, W., Feldhusen, J., and Grote, K.-H., 1984, Engineering Design: A Systematic Approach, Springer-Verlag, London, UK.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E., 2011, “Scikit-Learn: Machine Learning in Python,” J. Mach. Learn. Res., 12, pp. 2825–2830.
Fuge, M., and Agogino, A., 2014, “User Research Methods for Development Engineering: A Study of Method Usage With IDEO's HCD Connect,” ASME International Design Engineering Technical Conferences, Buffalo, NY, August 17–20.
Friedman, J., Hastie, T., and Tibshirani, R., 2008, “Sparse Inverse Covariance Estimation With the Graphical Lasso,” Biostatistics, 9(3), pp. 432–441. [CrossRef] [PubMed]
Herlocker, J. L., Konstan, J. A., Terveen, L. G., and Riedl, J. T., 2004, “Evaluating Collaborative Filtering Recommender Systems,” ACM Trans. Inf. Syst., 22(1), pp. 5–53. [CrossRef]
Wu, W., He, L., and Yang, J., 2012, “Evaluating Recommender Systems,” 7th International Conference on Digital Information Management (ICDIM), pp. 56–61. [CrossRef]
Gordon, M., and Pathak, P., 1999, “Finding Information on the World Wide Web: The Retrieval Effectiveness of Search Engines,” Inf. Process. Manage., 35(2), pp. 141–180. [CrossRef]
Wood, W. H., and Agogino, A. M., 2005, “Decision-Based Conceptual Design: Modeling and Navigating Heterogeneous Design Spaces,” ASME J. Mech. Des., 127(1), pp. 2–11. [CrossRef]
Hernandez, N. V., Schmidt, L. C., and Okudan, G. E., 2013, “Systematic Ideation Effectiveness Study of TRIZ,” ASME J. Mech. Des., 135(10), p. 101009. [CrossRef]
Kalyanasundaram, V., and Lewis, K., 2014, “A Function Based Approach for Product Integration,” ASME J. Mech. Des., 136(4), p. 041002. [CrossRef]
Srivastava, J., and Shu, L. H., 2013, “Affordances and Product Design to Support Environmentally Conscious Behavior,” ASME J. Mech. Des., 135(10), p. 101006. [CrossRef]
Fuge, M., Tee, K., Agogino, A., and Maton, N., 2014, “Analysis of Collaborative Design Networks: A Case Study of OpenIDEO,” ASME J. Comput. Inf. Sci. Eng., 14(2), p. 021009. [CrossRef]


Grahic Jump Location
Fig. 1

HCD Connect users use different methods with different frequencies. Error bars represent 95% confidence bounds around the frequency estimates, calculated using bootstrap resampling. The gray line represents the average method frequency (≈14%).

Grahic Jump Location
Fig. 2

Each case page on HCD Connect contains a textual description about the design problem (1), as well as contextual labels such as location (2), focus area (3), and user occupation (4)

Grahic Jump Location
Fig. 3

We performed spectral clustering on the 39 × 39 method covariance matrix revealing groups of methods that covary together. Lighter tones represent low covariance, while darker tones represent high covariance. The different hues denote different clusters, with a dark gray box around each cluster of methods. The clusters found by spectral clustering accurately reflect the expert-given categories used by IDEO in their HCD Toolkit; from left to right, the boxes on the diagonal correspond to “Deliver,” “Hear,” and “Create” methods, respectively.

Grahic Jump Location
Fig. 4

Precision plotted as a function of recall. The higher the AUC, the better the algorithm's performance.

Grahic Jump Location
Fig. 5

The area under the precision–recall curve (AUC) across the models. The error bars represent the 95% empirical confidence bounds about the median AUC for each method, calculated using bootstrap resampling. The hybrid and collaborative filtering models perform substantially better than the popularity baseline. The Random Forest classifier produces a detectable, but small, improvement over the popularity baseline.



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In