Research Papers: Design Automation

Behavioral Experimentation and Game Theory in Engineering Systems Design

[+] Author and Article Information
Zhenghui Sha

School of Mechanical Engineering,
Purdue University,
West Lafayette, IN 47907

Karthik N. Kannan

Associate Professor
Krannert School of Management,
Purdue University,
West Lafayette, IN 47907

Jitesh H. Panchal

Assistant Professor
School of Mechanical Engineering,
Purdue University,
West Lafayette, IN 47907

Since the number of tries has been scaled, one unit means 10 tries in low cost setting and 20 tries in high cost setting.

1Corresponding author.

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received October 1, 2014; final manuscript received January 15, 2015; published online March 5, 2015. Assoc. Editor: Bernard Yannou.

J. Mech. Des 137(5), 051405 (May 01, 2015) (10 pages) Paper No: MD-14-1665; doi: 10.1115/1.4029767 History: Received October 01, 2014; Revised January 15, 2015; Online March 05, 2015

Game-theoretic models have been used to analyze design problems ranging from multi-objective design optimization to decentralized design and from design for market systems (DFMS) to policy design. However, existing studies are primarily analytical in nature, which start with a number of assumptions about the individual decisions, the information available to the players, and the solution concept (generally, the Nash equilibrium). There is a lack of studies related to engineering design, which rigorously evaluate the validity of these assumptions or that of the predictions from the models. Hence, the usefulness of these models to realistic engineering systems design has been severely limited. In this paper, we take a step toward addressing this gap. Using an example of crowdsourcing for engineering design, we illustrate how the analytical game-theoretic models and behavioral experimentation can be synergistically used to gain a better understanding of design situations. Analytical models describe what players with assumed behaviors and cognitive capabilities would do under specified conditions, and the behavioral experiments shed light on how individuals actually behave. The paper contributes to the design literature in multiple ways. First, to the best of our knowledge, it is a first attempt at integrated theoretical and experimental game-theoretic analysis in design. We illustrate how the analytical models can be used to design behavioral experiments, which, in turn, can be used to estimate parameters, refine models, and inform further development of the theory. Second, we present a simple experiment to understand behaviors of individuals in a design crowdsourcing problem. The results of the experiment show new insights on using crowdsourcing contests for design.

Copyright © 2015 by ASME
Your Session has timed out. Please sign back in to continue.


Vincent, T. L., 1983, “Game Theory as a Design Tool,” J. Mech. Transm. Autom. Des., 105(2), pp. 165–170. [CrossRef]
Lewis, K., and Mistree, F., 1998, “Collaborative, Sequential and Isolated Decisions in Design,” ASME J. Mech. Des., 120(4), pp. 643–652. [CrossRef]
Xiao, A., Zeng, S., Allen, J. K., Rosen, D. W., and Mistree, F., 2005, “Collaborative Multidisciplinary Decision Making Using Game Theory and Design Capability Indices,” Res. Eng. Des., 16(1–2), pp. 57–72. [CrossRef]
Chanron, V., and Lewis, K., 2005, “A Study of Convergence in Decentralized Design Processes,” Res. Eng. Des., 16(3), pp. 133–145. [CrossRef]
Devendorf, E., and Lewis, K., 2011, “The Impact of Process Architecture on Equilibrium Stability in Distributed Design,” ASME J. Mech. Des., 133(10), p. 101001. [CrossRef]
Ciucci, F., Honda, T., and Yang, M. C., 2011, “An Information-Passing Strategy for Achieving Pareto Optimality in the Design of Complex Systems,” Res. Eng. Des., 23(1), pp. 71–83. [CrossRef]
Fernández, M. G., Panchal, J. H., Allen, J. K., and Mistree, F., 2005, “Concise Interactions and Effective Management of Shared Design Spaces: Moving Beyond Strategic Collaboration Towards Co-Design,” ASME Paper No. DETC2005-85381. [CrossRef]
Rao, S. S., and Freiheit, T. I., 1991, “A Modified Game Theory Approach to Multiobjective Optimization,” ASME J. Mech. Des., 113(3), pp. 286–291. [CrossRef]
Takai, S., 2010, “A Game-Theoretic Model of Collaboration in Engineering Design,” ASME J. Mech. Des., 132(5), p. 051005. [CrossRef]
Frischknecht, B. D., Whitefoot, K., and Papalambros, P. Y., 2010, “On the Suitability of Econometric Demand Models in Design for Market Systems,” ASME J. Mech. Des., 132(12), p. 121007. [CrossRef]
Taha, A. F., and Panchal, J. H., 2014, “Multilevel Decision-Making in Decentralized Energy Systems With Multiple Technologies and Uncertainty,” IEEE Trans. Syst. Man Cybern., 44(7), pp. 894–907. [CrossRef]
Taha, A. F., Hachem, N. A., and Panchal, J. H., 2014, “A Quasi-Feed-In-Tariff Policy Formulation in Micro-Grids: A Bi-Level Multi-Period Approach,” Energy Policy, 71, pp. 63–75. [CrossRef]
Kahneman, D., Slovic, P., and Tversky, A., 1982, Judgment Under Uncertainty: Heuristics and Biases, Cambridge University Press, New York. [CrossRef]
Simon, H., 1996, The Sciences of the Artificial, The MIT Press, Cambridge, MA.
Papalambros, P. Y., 2010, “The Human Dimension,” ASME J. Mech. Des., 132(5), p. 050201. [CrossRef]
Gurnani, A., and Lewis, K., 2008, “Collaborative, Decentralized Engineering Design at the Edge of Rationality,” ASME J. Mech. Des., 130(12), p. 121101. [CrossRef]
Franssen, M., and Bucciarelli, L. L., 2005, “On Rationality in Engineering Design,” ASME J. Mech. Des., 126(6), pp. 945–949. [CrossRef]
Boudreau, K. J., Lacetera, N., and Lakhani, K. R., 2011, “Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis,” Manage. Sci., 57(5), pp. 843–863. [CrossRef]
Seda Ertaç, S. M., 2010, Experimental Economics, SAGE Publications, Thousand Oaks, CA, pp. 873–883. [PubMed] [PubMed]
Ariely, D., and Norton, M. I., 2007, “Psychology and Experimental Economics,” Curr. Dir. Psychol. Sci., 16(6), pp. 336–339. [CrossRef]
Simon, H. A., 1956, “Rational Choice and the Structure of the Environment,” Psychol. Rev., 63(2), pp. 129–138. [CrossRef] [PubMed]
Holt, C. A., 2007, Markets, Games, and Strategic Behavior, Addison Wesley, Boston, MA.
Howe, J., 2008, Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business, Crown Business, New York.
LocalMotors, 2014, “Local Motors Challenges.” Available at: https://localmotors.com/challenges/
Innocentive, 2014, “Innocentive.” Available at: http://www.innocentive.com/
Quirky, 2014, “Quirky.” Available at: https://www.quirky.com/
TopCoder, 2014, “Topcoder.” Available at: http://www.topcoder.com/
Salek, M., Bachrach, Y., and Key, P., 2013, “Hotspotting: A Probabilistic Graphical Model for Image Object Localization Through Crowdsourcing,” Twenty-Seventh AAAI Conference on Artificial Intelligence, Bellevue, WA, July 14–18, pp. 1156–1162.
Lin, C. H., Mausam, and Weld, D. S., 2012, “Crowdsourcing Control: Moving Beyond Multiple Choice,” Twenty-Eighth Conference on Uncertainty in Artificial Intelligence, Catalina Island, CA, Aug. 14–18, pp. 491–500.
Raykar, V. C., and Yu, S., 2012, “Eliminating Spammers and Ranking Annotators for Crowdsourced Labeling Tasks,” J. Mach. Learn. Res., 13(1), pp. 491–518.
Baba, Y., and Kashima, H., 2013, “Statistical Quality Estimation for General Crowdsourcing Tasks,” 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’13, ACM, pp. 554–562. [CrossRef]
Green, M., Seepersad, C. C., and Holtta-Otto, K., 2014, “Crowd-Sourcing the Evaluation of Creativity in Conceptual Design: A Pilot Study,” ASME Paper No. DETC2014–34434. [CrossRef]
Gerth, R. J., Burnap, A., and Papalambros, P., 2012, “Crowdsourcing: A Primer and Its Implications for Systems Engineering,” 2012 NDIA Ground Vehicle Systems Engineering and Technology Symposium, Troy, MI, Aug. 14–16.
Kudrowitz, B. M., and Wallace, D., 2013, “Assessing the Quality of Ideas From Prolific, Early-Stage Product Ideation,” J. Eng. Des., 24(2), pp. 120–139. [CrossRef]
DARPA, 2014, “DARPA Adaptive Vehicle Make (AVM).” Available at: http://www.darpa.mil/NewsEvents/Releases/2013/04/22.aspx
Corchón, L. C., 2007, “The Theory of Contests: A Survey,” Rev. Econ. Des., 11(2), pp. 69–100. [CrossRef]
Terwiesch, C., and Xu, Y., 2008, “Innovation Contests, Open Innovation, and Multiagent Problem Solving,” Manage. Sci., 54(9), pp. 1529–1543. [CrossRef]
Skaperdas, S., 1996, “Contest Success Functions,” Economic Theory, 7(2), pp. 283–290. [CrossRef]
Chen, W., Hoyle, C., and Wassenaar, H. J., 2013, Decision-Based Design: Integrating Consumer Preferences into Engineering Design, Springer, New York.
Skaperdas, S., 1998, “On the Formation of Alliances in Conflict and Contests,” Public Choice, 96(1/2), pp. 25–42. [CrossRef]
Baldwin, C. Y., and Clark, K. B., 2000, Design Rules (The Power of Modularity, Vol. 1), The MIT Press, Cambridge, MA.
Papalambros, P. Y., and Wilde, D. J., 2000, Principles of Optimal Design: Modeling and Computation, 2nd ed., Cambridge University Press, New York. [CrossRef]
Andreoni, J., and Croson, R., 2008, “Partners Versus Strangers: Random Rematching in Public Goods Experiments,” Handbook of Experimental Economics Results, Vol. 1, Elsevier, Amsterdam, The Netherlands, pp. 776–783. [CrossRef]
Fischbacher, U., 2007, “z-Tree: Zurich Toolbox for Ready-Made Economic Experiments,” Exp. Econ., 10(2), pp. 171–178. [CrossRef]
Tversky, A., and Kahneman, D., 1974, “Judgment Under Uncertainty: Heuristics and Biases,” Science, 185(4157), pp. 1124–1131. [CrossRef] [PubMed]
Simpson, T. W., and Martins, J. R. R. A., 2011, “Multidisciplinary Design Optimization for Complex Engineered Systems: Report From a National Science Foundation Workshop,” ASME J. Mech. Des., 133(10), p. 101002. [CrossRef]
Camerer, C. F., 2003, Behavioral Game Theory: Experiments in Strategic Interaction, Princeton University Press, Princeton, NJ.
Loewenstein, G., 1999, “Experimental Economics From the Vantage-Point of Behavioural Economics,” Econ. J., 109(453), pp. F25–F34. [CrossRef]
Smith, V. L., 1976, “Experimental Economics: Induced Value Theory,” Am. Econ. Rev., 66(2), pp. 274–279.
Lichtenstein, S., and Slovic, P., 1973, “Response-Induced Reversals of Preference in Gambling: An Extended Replication in Las Vegas,” J. Exp. Psychol., 101(1), pp. 16–20. [CrossRef]
Lichtenstein, S., and Slovic, P., 1971, “Reversals of Preference Between Bids and Choices in Gambling Decisions,” J. Exp. Psychol., 89(1), pp. 46–55. [CrossRef]


Grahic Jump Location
Fig. 1

Relationship between parameters

Grahic Jump Location
Fig. 3

Relationship between number of tries and quality of solution for sessions 1 and 4 (low cost first)

Grahic Jump Location
Fig. 4

Relationship between number of tries and quality of solution for sessions 2 and 3 (high cost first)

Grahic Jump Location
Fig. 5

Predicted winning probability as a function of number of tries (e) for four scenarios

Grahic Jump Location
Fig. 2

Screenshot of the interface used by the participants




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In