Research Papers: Design Automation

Quantifying the Impact of Domain Knowledge and Problem Framing on Sequential Decisions in Engineering Design

[+] Author and Article Information
Murtuza Shergadwala

School of Mechanical Engineering,
Purdue University,
West Lafayette, IN 47907
e-mail: mshergad@purdue.edu

Ilias Bilionis

Assistant Professor
School of Mechanical Engineering,
Purdue University,
West Lafayette, IN 47907
e-mail: ibilion@purdue.edu

Karthik N. Kannan

Krannert School of Management,
Purdue University,
West Lafayette, IN 47907
e-mail: kkarthik@purdue.edu

Jitesh H. Panchal

Associate Professor
School of Mechanical Engineering,
Purdue University,
West Lafayette, IN 47907
e-mail: panchal@purdue.edu

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received March 4, 2018; final manuscript received June 3, 2018; published online July 24, 2018. Assoc. Editor: Harrison M. Kim.

J. Mech. Des 140(10), 101402 (Jul 24, 2018) (13 pages) Paper No: MD-18-1187; doi: 10.1115/1.4040548 History: Received March 04, 2018; Revised June 03, 2018

Many decisions within engineering systems design are typically made by humans. These decisions significantly affect the design outcomes and the resources used within design processes. While decision theory is increasingly being used from a normative standpoint to develop computational methods for engineering design, there is still a significant gap in our understanding of how humans make decisions within the design process. Particularly, there is lack of knowledge about how an individual's domain knowledge and framing of the design problem affect information acquisition decisions. To address this gap, the objective of this paper is to quantify the impact of a designer's domain knowledge and problem framing on their information acquisition decisions and the corresponding design outcomes. The objective is achieved by (i) developing a descriptive model of information acquisition decisions, based on an optimal one-step look ahead sequential strategy, utilizing expected improvement maximization, and (ii) using the model in conjunction with a controlled behavioral experiment. The domain knowledge of an individual is measured in the experiment using a concept inventory, whereas the problem framing is controlled as a treatment variable in the experiment. A design optimization problem is framed in two different ways: a domain-specific track design problem and a domain-independent function optimization problem (FOP). The results indicate that when the problem is framed as a domain-specific design task, the design solutions are better and individuals have a better state of knowledge about the problem, as compared to the domain-independent task. The design solutions are found to be better when individuals have a higher knowledge of the domain and they follow the modeled strategy closely.

Copyright © 2018 by ASME
Your Session has timed out. Please sign back in to continue.


Hazelrigg, G. A. , 1998, “ A Framework for Decision-Based Engineering Design,” ASME J. Mech. Des., 120(4), pp. 653–658. [CrossRef]
Marston, M. , and Mistree, F. , 1997, “ A Decision-Based Foundation for Systems Design: A Conceptual Exposition,” International Design Seminar Proceedings on Multimedia Technologies for Collaborative Design and Manufacturing (CIRP 1997), pp. 1–11.
Lewis, K. , Chen, W. , Schmidt, L. , and Chen, W. , eds., 2006, Decision Making in Engineering Design, ASME Press, New York.
Fischer, G. W. , 1979, “ Utility Models for Multiple Objective Decisions: Do They Accurately Represent Human Preferences?,” Decis. Sci., 10(3), pp. 451–479. [CrossRef]
Belton, V. , 1986, “ A Comparison of the Analytic Hierarchy Process and a Simple Multi-Attribute Value Function,” Eur. J. Oper. Res., 26(1), pp. 7–21. [CrossRef]
Gurnani, A. , and Lewis, K. , 2008, “ Collaborative, Decentralized Engineering Design at the Edge of Rationality,” ASME J. Mech. Des., 130(12), p. 121101. [CrossRef]
See, T.-K. , and Lewis, K. , 2006, “ A Formal Approach to Handling Conflicts in Multiattribute Group Decision Making,” ASME J. Mech. Des., 128(4), pp. 678–688. [CrossRef]
Wassenaar, H. J. , and Chen, W. , 2003, “ An Approach to Decision-Based Design With Discrete Choice Analysis for Demand Modeling,” ASME J. Mech. Des., 125(3), pp. 490–497. [CrossRef]
Thompson, S. C. , 2011, “ Rational Design Theory: A Decision-Based Foundation for Studying Design Methods,” Ph.D. thesis, Georgia Institute of Technology, Atlanta, GA. https://smartech.gatech.edu/handle/1853/39490
Moore, J. C. , and Whinston, A. B. , 1986, “ A Model of Decision-Making With Sequential Information-Acquisition (Part 1),” Decis. Support Syst., 2(4), pp. 285–307. [CrossRef]
Campanella, G. , and Ribeiro, R. A. , 2011, “ A Framework for Dynamic Multiple-Criteria Decision Making,” Decis. Support Syst., 52(1), pp. 52–60. [CrossRef]
Antle, J. M. , 1983, “ Sequential Decision Making in Production Models,” Am. J. Agric. Econ., 65(2), pp. 282–290. [CrossRef]
Chiesi, H. L. , Spilich, G. J. , and Voss, J. F. , 1979, “ Acquisition of Domain-Related Information in Relation to High and Low Domain Knowledge,” J. Verbal Learn. Verbal Behav., 18(3), pp. 257–273. [CrossRef]
Gao, S. , and Kvan, T. , 2004, “ An Analysis of Problem Framing in Multiple Settings,” Design Computing and Cognition '04, J. S. Gero , ed., Springer, Dordrecht, The Netherlands, pp. 117–134. [CrossRef]
Schön, D. A. , 1987, Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions, Jossey-Bass, San Francisco, CA.
Schön, D. A. , 1984, “ Problems, Frames and Perspectives on Designing,” Des. Stud., 5(3), pp. 132–136. [CrossRef]
Hestenes, D. , Wells, M. , Swackhamer, G. , Halloun, I. , Hake, R. , and Mosca, E. , 1995, “ Revised Force Concept Inventory,” PhysPort, accessed June 26, 2018, https://www.physport.org/assessments/assessment.cfm?I=5&A=FCI
Loch, C. H. , Terwiesch, C. , and Thomke, S. , 2001, “ Parallel and Sequential Testing of Design Alternatives,” Manage. Sci., 47(5), pp. 663–678. [CrossRef]
Simon, H. A. , 1990, Bounded Rationality, Palgrave Macmillan, London. [CrossRef]
Rasmussen, C. E. , and Williams, C. K. , 2006, Gaussian Processes for Machine Learning, MIT Press, Cambridge, UK.
Lucas, C. G. , Griffiths, T. L. , Williams, J. J. , and Kalish, M. L. , 2015, “ A Rational Model of Function Learning,” Psychon. Bull. Rev., 22(5), pp. 1193–1215. [CrossRef] [PubMed]
Wilson, A. G. , Dann, C. , Lucas, C. , and Xing, E. P. , 2015, “ The Human Kernel,” Advances in Neural Information Processing Systems 28, C. Cortes , N. D. Lawrence , D. D. Lee , M. Sugiyama , and R. Garnett , eds., Curran Associates, Red Hook, NY, pp. 2854–2862.
Borji, A. , and Itti, L. , 2013, “ Bayesian Optimization Explains Human Active Search,” Advances in Neural Information Processing Systems 26, C. J. C. Burges , L. Bottou , M. Welling , Z. Ghahramani , and K. Q. Weinberger , eds., Curran Associates, Red Hook, NY, pp. 55–63.
Hinson, J. M. , Jameson, T. L. , and Whitney, P. , 2003, “ Impulsive Decision Making and Working Memory,” J. Exp. Psychology: Learn., Memory, Cognit., 29(2), pp. 298–306. [CrossRef]
Bernardo, J. , Bayarri, M. , Berger, J. , Dawid, A. , Heckerman, D. , Smith, A. , and West, M. , 2011, “ Optimization Under Unknown Constraints,” Bayesian Statistics 9, Oxford University Press, Oxford, UK, pp. 229–256.
Metropolis, N. , Rosenbluth, A. W. , Rosenbluth, M. N. , Teller, A. H. , and Teller, E. , 1953, “ Equation of State Calculations by Fast Computing Machines,” J. Chem. Phys., 21(6), pp. 1087–1092. [CrossRef]
Patil, A. , Huard, D. , and Fonnesbeck, C. J. , 2010, “ PYMC: Bayesian Stochastic Modelling in Python,” J. Stat. Software, 35(4), pp. 1–81. [CrossRef]
Chi, M. T. , Feltovich, P. J. , and Glaser, R. , 1981, “ Categorization and Representation of Physics Problems by Experts and Novices,” Cognit. Sci., 5(2), pp. 121–152. [CrossRef]
Valerij, D. , 2013, “ Relationship Between Learning, Knowledge Creation and Organisational Performance,” Ann. Alexandru Ioan Cuza Univ.-Econ., 60(1), pp. 79–93. [CrossRef]
Shaughnessy, J. J. , and Zechmeister, E. B. , 1985, Research Methods in Psychology, Alfred A. Knopf, New York.
Eatwell, J. , Milgate, M. , and Newman, P. , 1987, The New Palgrave: A Dictionary of Economics, Macmillan, London. [CrossRef]
Wang, J. , and Bao, L. , 2010, “ Analyzing Force Concept Inventory With Item Response Theory,” Am. J. Phys., 78(10), pp. 1064–1070. [CrossRef]
Shergadwala, M. , Kannan, K. N. , and Panchal, J. H. , 2016, “ Understanding the Impact of Expertise on Design Outcome: An Approach Based on Concept Inventories and Item Response Theory,” ASME Paper No. DETC2016-59038.
Panchal, J. H. , and Szajnfarber, Z. , “ Experiments in Systems Engineering and Design Research,” Syst. Eng., 20(6), pp. 529–541. [CrossRef]
Shadish, W. , Cook, T. D. , and Campbell, D. T. , 2002, Experimental and Quasi-Experimental Designs for Generalized Causal Inference, Wadsworth Cengage Learning, Belmont, CA.


Grahic Jump Location
Fig. 1

Illustration of past research emphasis in decision-based design and the focus of this paper

Grahic Jump Location
Fig. 2

Illustration of a SIADM process. Decisions are highlighted in gray color. Rectangular nodes are information acquisition decisions and the outcome (diamond node) of the SIADM process is making the artifact decision.

Grahic Jump Location
Fig. 4

The user interface of track design game where constraint is specified TDPCS

Grahic Jump Location
Fig. 5

Scatter plots for H1*: (a) TDPCNS and (b) TDPCS

Grahic Jump Location
Fig. 3

Graphical illustration of SIADM model at step t. Parameters λ, b, l, and s are inferred by the individual. Parameters μb, σb, μλ, σλ, and σ are a part of an individual's type θ.

Grahic Jump Location
Fig. 7

Scatter plots for H3*: (a) TDPCNS and (b) TDPCS

Grahic Jump Location
Fig. 6

Scatter plots for H2*: (a) TDPCNS and (b) TDPCS



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In