0
Research Papers: Variability/Uncertainty in D3

Mining Process Heuristics From Designer Action Data via Hidden Markov Models

[+] Author and Article Information
Christopher McComb

Mem. ASME
Department of Mechanical Engineering,
Carnegie Mellon University,
Pittsburgh, PA 15232
e-mail: ccm@cmu.edu

Jonathan Cagan

Fellow ASME
Department of Mechanical Engineering,
Carnegie Mellon University,
Pittsburgh, PA 15232
e-mail: cagan@cmu.edu

Kenneth Kotovsky

Department of Psychology,
Carnegie Mellon University,
Pittsburgh, PA 15232
e-mail: kotovsky@cmu.edu

Contributed by the Design Theory and Methodology Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received February 20, 2017; final manuscript received June 5, 2017; published online October 2, 2017. Assoc. Editor: Harrison M. Kim.

J. Mech. Des 139(11), 111412 (Oct 02, 2017) (12 pages) Paper No: MD-17-1161; doi: 10.1115/1.4037308 History: Received February 20, 2017; Revised June 05, 2017

Configuration design problems, characterized by the assembly of components into a final desired solution, are common in engineering design. Various theoretical approaches have been offered for solving configuration type problems, but few studies have examined the approach that humans naturally use to solve such problems. This work applies data-mining techniques to quantitatively study the processes that designers use to solve configuration design problems. The guiding goal is to extract beneficial design process heuristics that are generalizable to the entire class of problems. The extraction of these human problem-solving heuristics is automated through the application of hidden Markov models to the data from two behavioral studies. Results show that designers proceed through four procedural states in solving configuration design problems, roughly transitioning from topology design to shape and parameter design. High-performing designers are distinguished by their opportunistic tuning of parameters early in the process, enabling a more effective and nuanced search for solutions.

FIGURES IN THIS ARTICLE
<>
Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.

References

Mittal, S. , and Frayman, F. , 1989, “ Towards a Generic Model of Configuration Tasks,” 11th International Joint Conference on Artificial Intelligence (IJCAI), Detroit, MI, Aug. 20–26, Vol. 2, pp. 1395–1401.
McComb, C. , Cagan, J. , and Kotovsky, K. , 2015, “ Rolling With the Punches: An Examination of Team Performance in a Design Task Subject to Drastic Changes,” Des. Stud., 36(1), pp. 99–121. [CrossRef]
McComb, C. , Cagan, J. , and Kotovsky, K. , 2017, “ Optimizing Design Teams Based on Problem Properties: Computational Team Simulations and an Applied Empirical Test,” ASME J. Mech. Des., 139(4), p. 041101. [CrossRef]
Chandrasekaran, B. , 1990, “ Design Problem Solving: A Task Analysis,” AI Mag., 11(4), pp. 59–71.
Wielinga, B. , and Schreiber, G. , 1997, “ Configuration-Design Problem Solving,” IEEE Expert Intell. Syst. Appl., 12(2), pp. 49–56.
McComb, C. , Cagan, J. , and Kotovsky, K. , 2015, “ Lifting the Veil: Drawing Insights About Design Teams From a Cognitively-Inspired Computational Model,” Des. Stud., 40, pp. 119–142. [CrossRef]
McComb, C. , Cagan, J. , and Kotovsky, K. , 2017, “ Capturing Human Sequence-Learning Abilities in Configuration Design Tasks Through Markov Chains,” ASME J. Mech. Des., 139(9), p. 091101. [CrossRef]
Kim, M. J. , and Maher, M. L. , 2008, “ The Impact of Tangible User Interfaces on Spatial Cognition During Collaborative Design,” Des. Stud., 29(3), pp. 222–253. [CrossRef]
Snider, C. , Dekoninck, E. A. , and Culley, S. J. , 2014, “ The Appearance of Creative Behavior in Later Stage Design Processes,” Int. J. Des. Creativity Innovation, 2(1), pp. 1–19. [CrossRef]
Visser, W. , 1994, “ Organisation of Design Activities: Opportunistic, With Hierarchical Episodes,” Interact. Comput., 6(3), pp. 239–274. [CrossRef]
Guindon, R. , 1990, “ Designing the Design Process: Exploiting Opportunistic Thoughts,” Hum.-Comput. Interact., 5(2), pp. 305–344. [CrossRef]
Bender, B. , and Blessing, L. , 2004, “ On the Superiority of Opportunistic Design Strategies During Early Embodiment Design,” Eighth International Design Conference (DESIGN), Dubrovnik, Croatia, May 18–21, pp. 1–6.
Durbin, R. , Eddy, S. R. , Krogh, A. , and Mitchison, G. , 1998, Biological Sequence Analysis, Cambridge University Press, Cambridge, UK. [CrossRef]
Baum, L. E. , and Petrie, T. , 1966, “ Statistical Inference for Probabilistic Functions of Finite State Markov Chains,” Ann. Math. Stat., 37(6), pp. 1554–1563. [CrossRef]
Baum, L. E. , and Eagon, J. A. , 1967, “ An Inequality With Applications to Statistical Estimation for Probabilistic Functions of Markov Processes and to a Model for Ecology,” Bull. Am. Math. Soc., 73(3), pp. 360–364. [CrossRef]
Baum, L. E. , and Sell, G. , 1968, “ Growth Transformations for Functions on Manifolds,” Pac. J. Math., 27(2), pp. 211–227. [CrossRef]
Baum, L. E. , Petrie, T. , Soules, G. , and Weiss, N. , 1970, “ A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains,” Ann. Math. Stat., 41(1), pp. 164–171. [CrossRef]
Jelinek, F. , Bahl, L. , and Mercer, R. , 1975, “ Design of a Linguistic Statistical Decoder for the Recognition of Continuous Speech,” IEEE Trans. Inf. Theory, 21(3), pp. 250–256. [CrossRef]
Krogh, A. , Brown, M. , Mian, I. S. , Sjölander, K. , and Haussler, D. , 1994, “ Hidden Markov Models in Computational Biology: Applications to Protein Modeling,” J. Mol. Biol., 235(5), pp. 1501–1531. [CrossRef] [PubMed]
Gonzalez, A. M. , Roque, A. M. S. , and Garcia-Gonzalez, J. , 2005, “ Modeling and Forecasting Electricity Prices With Input/Output Hidden Markov Models,” IEEE Trans. Power Syst., 20(1), pp. 13–24. [CrossRef]
White, B. , Blaylock, N. , and Bölöni, L. , 2009, “ Analyzing Team Actions With Cascading HMM,” The Florida Artificial Intelligence Research Society Conference (FLAIRS), Sanibel Island, FL, May 19–21, pp. 129–134.
Tenison, C. , and Anderson, J. R. , 2016, “ Modeling the Distinct Phases of Skill Acquisition,” J. Exp. Psychol. Learn. Mem. Cognit., 42(5), pp. 749–767. [CrossRef]
Anderson, J. R. , and Fincham, J. M. , 2014, “ Discovering the Sequential Structure of Thought,” Cognit. Sci., 38(2), pp. 322–352. [CrossRef]
Atman, C. J. , Adams, R. S. , Cardella, M. E. , Turns, J. , Mosborg, S. , and Saleem, J. , 2007, “ Engineering Design Processes: A Comparison of Students and Expert Practitioners,” J. Eng. Educ., 96(4), pp. 359–379. [CrossRef]
Stempfle, J. , and Badke-Schaub, P. , 2002, “ Thinking in Design Teams—An Analysis of Team Communication,” Des. Stud., 23(5), pp. 473–496. [CrossRef]
Goldschmidt, G. , and Rodgers, P. A. , 2013, “ The Design Thinking Approaches of Three Different Groups of Designers Based on Self-Reports,” Des. Stud., 34(4), pp. 454–471. [CrossRef]
Kotovsky, K. , Hayes, J. , and Simon, H. A. , 1985, “ Why Are Some Problems Hard? Evidence From Tower of Hanoi,” Cognit. Psychol., 17(2), pp. 248–294. [CrossRef]
McComb, C. , Jonathan, C. , and Kotovsky, K. , 2017, “ Validating a Tool for Predicting Problem-Specific Optimized Team Characteristics,” ASME Paper No. DETC2017-67430.
Dempster, A. , Laird, N. M. , and Rubin, D. B. , 1977, “ Maximum Likelihood From Incomplete Data Via the EM Algorithm,” J. R. Stat. Soc.: Ser. B, 39(1), pp. 1–38.
Stratonovich, R. L. , 1960, “ Conditional Markov Processes,” Theory Probab. Its Appl., 5(2), pp. 156–178. [CrossRef]
Arlot, S. , and Celisse, A. , 2010, “ A Survey of Cross-Validation Procedures for Model Selection,” Stat. Surv., 4, pp. 40–79. [CrossRef]
Borst, J. P. , and Anderson, J. R. , 2013, “ Discovering Processing Stages by Combining EEG With Hidden Markov Models,” 35th Annual Conference of the Cognitive Science Society (CogSci), Berlin, July 31–Aug. 3, pp. 221–226.
McComb, C. , Cagan, J. , and Kotovsky, K. , 2017, “ Eliciting Configuration Design Heuristics With Hidden Markov Models,” International Conference on Engineering Design (ICED), Vancouver, BC, Canada, Aug. 21–25, Paper No. 162.

Figures

Grahic Jump Location
Fig. 1

Example of a hidden Markov model with three states and three emission tokens

Grahic Jump Location
Fig. 2

Example truss operation sequences

Grahic Jump Location
Fig. 3

Example cooling system operation sequence with shading indicating room temperatures

Grahic Jump Location
Fig. 4

Testing log-likelihood on truss study data for models with increasing number of hidden states (error bars show ±1 S.E.)

Grahic Jump Location
Fig. 5

Transition (left) and emission (right) matrices for four-state hidden Markov model based on the data from the truss study

Grahic Jump Location
Fig. 6

Visual representation of four-state hidden Markov model based on data from truss study

Grahic Jump Location
Fig. 7

Testing log-likelihood on cooling system study data for models with increasing number of hidden states (error bars show ±1 S.E.)

Grahic Jump Location
Fig. 8

Transition (left) and emission (right) matrices for four-state hidden Markov model based on the data from the cooling system study

Grahic Jump Location
Fig. 9

Visual representation of four-state hidden Markov model based on the data from the cooling system study

Grahic Jump Location
Fig. 10

Visual representation of four-state hidden Markov model for high-performing participants from the truss study. Model for low-performing participants is structurally identical to the aggregate model (Fig. 6).

Grahic Jump Location
Fig. 11

Comparison of transmission/emission matrices for the truss study trained on: (a) high-performing data and (b) low-performing data

Grahic Jump Location
Fig. 12

Visual representation of four-state hidden Markov model for high-performing participants from the cooling systems design study. Model for low-performing participants is structurally identical to aggregate model (Fig. 9).

Grahic Jump Location
Fig. 13

Comparison of transmission/emission matrices for the cooling system study trained on: (a) high-performing data and (b) low-performing data

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In