Graphical Abstract Figure
Abstract
A novel approach for computational agents to learn proficient behavior in engineering configuration design that is inspired by human learning is introduced in this work. The learning proficient simulated annealing design agents (LPSADA) begin as different proficiency designers and are explicitly modeled to mimic the design behavior and performance of different proficiency human designers. A learning methodology, which is inspired by human learning, is introduced to update the characteristics of the agents that dictate their behavior. The methods are designed to change their behavioral characteristics based on their experience, including a non-deterministic reinforcement learning algorithm. Results show that the lower-proficiency agents successfully change their behavior to act more like high-proficiency designers. These behavior changes are shown to increase the performance of the lower-proficiency agents to the levels of high-proficiency human designers. In sum, the learning methodology that is introduced is shown to allow lower-proficiency agents to become higher-proficiency designers.
References
1.
Brownell
, E.
, Cagan
, J.
, and Kotovsky
, K.
, 2023
, “A Computational Model of Human Proficiency in Engineering Configuration Design
,” ASME J. Mech. Des.
, 145
(10
), p. 101703
. 2.
McComb
, C.
, Cagan
, J.
, and Kotovsky
, K.
, 2015
, “Lifting the Veil: Drawing Insights About Design Teams From a Cognitively-Inspired Computational Model
,” Des. Stud.
, 40
, pp. 119
–142
. 3.
Bass
, B. M.
, 1980
, “Team Productivity and Individual Member Competence
,” Small Group Res.
, 11
(4
), pp. 431
–504
. 4.
Jin
, Y.
, and Lu
, S. C.-Y.
, 2004
, “Agent Based Negotiation for Collaborative Design Decision Making
,” CIRP Ann.
, 53
(1
), pp. 121
–124
. 5.
Campbell
, M. I.
, Cagan
, J.
, and Kotovsky
, K.
, 1999
, “A-Design: An Agent-Based Approach to Conceptual Design in a Dynamic Environment
,” Res. Eng. Des.
, 11
(3
), pp. 172
–192
. 6.
Singh
, H.
, Cascini
, G.
, and McComb
, C.
, 2021
, “Comparing Design Outcomes Achieved by Teams of Expert and Novice Designers Through Agent-Based Simulation
,” Proc. Des. Soc.
, 1
, pp. 661
–670
. 7.
Soria Zurita
, N. F.
, Colby
, M. K.
, Tumer
, I. Y.
, Hoyle
, C.
, and Tumer
, K.
, 2018
, “Design of Complex Engineered Systems Using Multi-agent Coordination
,” ASME J. Comput. Inf. Sci. Eng.
, 18
(1
), p. 011003
. 8.
Manion
, C.
, Soria
, N. F.
, Tumer
, K.
, Hoyle
, C.
, and Tumer
, I. Y.
, 2016
, “Designing a Self-replicating Robotic Manufacturing Factory
,” ASME 2015 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
, Boston, MA
, Aug. 2–5
.9.
Dimeas
, A. L.
, and Hatziargyriou
, N. D.
, 2005
, “Operation of a Multiagent System for Microgrid Control
,” IEEE Trans. Power Syst.
, 20
(3
), pp. 1447
–1455
. 10.
Perisic
, M. M.
, Martinec
, T.
, Storga
, M.
, and Gero
, J. S.
, 2019
, “A Computational Study of the Effect of Experience on Problem/Solution Space Exploration in Teams
,” Proc. Des. Soc. Int. Conf. Eng. Des.
, 1
(1
), pp. 11
–20
. 11.
Hulse
, D.
, Tumer
, K.
, Hoyle
, C.
, and Tumer
, I.
, 2019
, “Modeling Multidisciplinary Design With Multiagent Learning
,” Artif. Intell. Eng. Des. Anal. Manuf.
, 33
(1
), pp. 85
–99
. 12.
Singh
, V.
, Dong
, A.
, and Gero
, J. S.
, 2013
, “Social Learning in Design Teams: The Importance of Direct and Indirect Communications
,” Artif. Intell. Eng. Des. Anal. Manuf.
, 27
(2
), pp. 167
–182
. 13.
Tumer
, K.
, Agogino
, A. K.
, and Wolpert
, D. H.
, 2002
, “Learning Sequences of Actions in Collectives of Autonomous Agents
,” AAMAS‘02
, Bologna, Italy
, July 15–19
, pp. 378
–385
.14.
Sutton
, R. S.
, and Barto
, A. G.
, 2018
, Reinforcement Learning, Second Edition: An Introduction
, MIT Press
, Cambridge, MA
.15.
Cagan
, J.
, and Kotovsky
, K.
, 1997
, “Simulated Annealing and the Generation of the Objective Function: A Model of Learning During Problem Solving
,” Comput. Intell.
, 13
(4
), pp. 534
–581
. 16.
Shteingart
, H.
, and Loewenstein
, Y.
, 2014
, “Reinforcement Learning and Human Behavior
,” Curr. Opin. Neurobiol.
, 25
, pp. 93
–98
. 17.
Schultz
, W.
, Dayan
, P.
, and Montague
, P. R.
, 1997
, “A Neural Substrate of Prediction and Reward
,” Science
, 275
(5306
), pp. 1593
–1599
. 18.
Gershman
, S. J.
, and Daw
, N. D.
, 2017
, “Reinforcement Learning and Episodic Memory in Humans and Animals: An Integrative Framework
,” Annu. Rev. Psychol.
, 68
(1
), pp. 101
–128
. 19.
Regenwetter
, L.
, Nobari
, A. H.
, and Ahmed
, F.
, 2022
, “Deep Generative Models in Engineering Design: A Review
,” ASME J. Mech. Des.
, 144
(7
), p. 071704
. 20.
Ororbia
, M. E.
, and Warn
, G. P.
, 2023
, “Design Synthesis of Structural Systems as a Markov Decision Process Solved With Deep Reinforcement Learning
,” ASME J. Mech. Des.
, 145
(6
), p. 061701
. 21.
Lee
, X. Y.
, Balu
, A.
, Stoecklein
, D.
, Ganapathysubramanian
, B.
, and Sarkar
, S.
, 2019
, “A Case Study of Deep Reinforcement Learning for Engineering Design: Application to Microfluidic Devices for Flow Sculpting
,” ASME J. Mech. Des.
, 141
(11
), p. 111401
. 22.
Raina
, A.
, Cagan
, J.
, and McComb
, C.
, 2023
, “Learning to Design Without Prior Data: Discovering Generalizable Design Strategies Using Deep Learning and Tree Search
,” ASME J. Mech. Des.
, 145
(3
), p. 031402
. 23.
Raina
, A.
, McComb
, C.
, and Cagan
, J.
, 2019
, “Learning to Design From Humans: Imitating Human Designers Through Deep Learning
,” ASME J. Mech. Des.
, 141
(11
), p. 111102
. 24.
Caputo
, C.
, and Cardin
, M.-A.
, 2022
, “Analyzing Real Options and Flexibility in Engineering Systems Design Using Decision Rules and Deep Reinforcement Learning
,” ASME J. Mech. Des.
, 144
(2
), p. 021705
. 25.
Chen
, Q.
, Heydari
, B.
, and Moghaddam
, M.
, 2021
, “Leveraging Task Modularity in Reinforcement Learning for Adaptable Industry 4.0 Automation
,” ASME J. Mech. Des.
, 143
(7
), p. 071701
. 26.
Chen
, Q.
, and Heydari
, B.
, 2022
, “Dynamic Resource Allocation in Systems-of-Systems Using a Heuristic-Based Interpretable Deep Reinforcement Learning
,” ASME J. Mech. Des.
, 144
(9
), p. 091711
. 27.
Raina
, A.
, Cagan
, J.
, and McComb
, C.
, 2019
, “Transferring Design Strategies From Human to Computer and Across Design Problems
,” ASME J. Mech. Des.
, 141
(11
), p. 114501
. 28.
Botvinick
, M.
, Ritter
, S.
, Wang
, J. X.
, Kurth-Nelson
, Z.
, Blundell
, C.
, and Hassabis
, D.
, 2019
, “Reinforcement Learning, Fast and Slow
,” Trends Cogn. Sci.
, 23
(5
), pp. 408
–422
. 29.
Mitchell
, T. M.
, Utgoff
, P. E.
, and Banerji
, R.
, 1983
, “Learning by Experimentation: Acquiring and Refining Problem-Solving Heuristics,” Machine Learning: An Artificial Intelligence Approach
, R. S.
Michalski
, J. G.
Carbonell
, and T. M.
Mitchell
, eds., Springer
, Berlin
, pp. 163
–190
.30.
Brownell
, E.
, Cagan
, J.
, and Kotovsky
, K.
, 2021
, “Only as Strong as the Strongest Link: The Relative Contribution of Individual Team Member Proficiency in Configuration Design
,” ASME J. Mech. Des.
, 143
(8
), p. 081402
. 31.
Cross
, N.
, 2004
, “Expertise in Design: An Overview
,” Des. Stud.
, 25
(5
), pp. 427
–441
. 32.
Cross
, N.
, 2018
, “Expertise in Professional Design,” The Cambridge Handbook of Expertise and Expert Performance
, K. A.
Ericsson
, R. R.
Hoffman
, A.
Kozbelt
, and A. M.
Williams
, eds., Cambridge University Press
, Cambridge, UK
, pp. 372
–388
.33.
Ahmed
, S.
, Wallace
, K. M.
, and Blessing
, L. T.
, 2003
, “Understanding the Differences Between How Novice and Experienced Designers Approach Design Tasks
,” Res. Eng. Des.
, 14
(1
), pp. 1
–11
. 34.
Puentes
, L.
, Cagan
, J.
, and McComb
, C.
, 2021
, “Data-Driven Heuristic Induction From Human Design Behavior
,” ASME J. Comput. Inf. Sci. Eng.
, 21
(2
), p. 024501
. 35.
McComb
, C.
, Cagan
, J.
, and Kotovsky
, K.
, 2017
, “Mining Process Heuristics From Designer Action Data Via Hidden Markov Models
,” ASME J. Mech. Des.
, 139
(11
), p. 111412
. 36.
McComb
, C.
, Cagan
, J.
, and Kotovsky
, K.
, 2017
, “Optimizing Design Teams Based on Problem Properties: Computational Team Simulations and an Applied Empirical Test
,” ASME J. Mech. Des.
, 139
(4
), p. 041101
. 37.
Schön
, D. A.
, 1988
, “Designing: Rules, Types and Worlds
,” Des. Stud.
, 9
(3
), pp. 181
–190
. 38.
Leibowitz
, N.
, Baum
, B.
, Enden
, G.
, and Karniel
, A.
, 2010
, “The Exponential Learning Equation as a Function of Successful Trials Results in Sigmoid Performance
,” J. Math. Psychol.
, 54
(3
), pp. 338
–340
. 39.
Ritter
, F. E.
, and Schooler
, L. J.
, 2001
, “The Learning Curve,” International Encyclopedia of the Social & Behavioral Sciences
, N. J.
Smelser
, and P. B.
Baltes
, eds., Elsevier
, New York
, pp. 8602
–8605
.40.
Thurstone
, L. L.
, 1919
, “The Learning Curve Equation
,” Psychol. Monogr.
, 26
(3
), pp. i
–51
. 41.
Estes
, W. K.
, 1950
, “Toward a Statistical Theory of Learning
,” Psychol. Rev.
, 57
(2
), pp. 94
–107
. 42.
Heathcote
, A.
, Brown
, S.
, and Mewhort
, D. J. K.
, 2000
, “The Power Law Repealed: The Case for an Exponential Law of Practice
,” Psychon. Bull. Rev.
, 7
(2
), pp. 185
–207
. 43.
Yilmaz
, S.
, Seifert
, C. M.
, and Gonzalez
, R.
, 2010
, “Cognitive Heuristics in Design: Instructional Strategies to Increase Creativity in Idea Generation
,” Artif. Intell. Eng. Des. Anal. Manuf.
, 24
(3
), pp. 335
–355
. 44.
Metcalfe
, J.
, and Wiebe
, D.
, 1987
, “Intuition in Insight and Noninsight Problem Solving
,” Mem. Cogn.
, 15
(3
), pp. 238
–246
. 45.
Siegler
, R. S.
, 2000
, “Unconscious Insights
,” Curr. Dir. Psychol. Sci.
, 9
(3
), pp. 79
–83
. 
