Research Papers: Design Automation

Quantifying the Resilience-Informed Scenario Cost Sum: A Value-Driven Design Approach for Functional Hazard Assessment

[+] Author and Article Information
Daniel Hulse, Christopher Hoyle

School of Mechanical, Industrial and
Manufacturing Engineering,
Oregon State University,
Corvallis, OR 97330

Kai Goebel

Tech Area Lead,
Discovery and Systems Health,
Intelligent Systems Division,
NASA Ames Research Center,
Moffett Field, CA 94035;
Adjunct Professor
Division of Operation and
Maintenance Engineering,
Luleå Technical University,
Luleå 97187, Sweden

Irem Y. Tumer

School of Mechanical, Industrial and
Manufacturing Engineering,
Oregon State University,
Corvallis, OR 97330

1Corresponding author.

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received June 28, 2018; final manuscript received September 10, 2018; published online December 20, 2018. Assoc. Editor: Nam H. Kim. This work is in part a work of the U.S. Government. ASME disclaims all interest in the U.S. Government's contributions.

J. Mech. Des 141(2), 021403 (Dec 20, 2018) (16 pages) Paper No: MD-18-1503; doi: 10.1115/1.4041571 History: Received June 28, 2018; Revised September 10, 2018

Complex engineered systems can carry risk of high failure consequences, and as a result, resilience—the ability to avoid or quickly recover from faults—is desirable. Ideally, resilience should be designed-in as early in the design process as possible so that designers can best leverage the ability to explore the design space. Toward this end, previous work has developed functional modeling languages which represent the functions which must be performed by a system and function-based fault modeling frameworks have been developed to predict the resulting fault propagation behavior of a given functional model. However, little has been done to formally optimize or compare designs based on these predictions, partially because the effects of these models have not been quantified into an objective function to optimize. The work described herein closes this gap by introducing the resilience-informed scenario cost sum (RISCS), a scoring function which integrates with a fault scenario-based simulation, to enable the optimization and evaluation of functional model resilience. The scoring function accomplishes this by quantifying the expected cost of a design's fault response using probability information, and combining this cost with design and operational costs such that it may be parameterized in terms of designer-specified resilient features. The usefulness and limitations of using this approach in a general optimization and concept selection framework are discussed in general, and demonstrated on a monopropellant system design problem. Using RISCS as an objective for optimization, the algorithm selects the set of resilient features which provides the optimal trade-off between design cost and risk. For concept selection, RISCS is used to judge whether resilient concept variants justify their design costs and make direct comparisons between different model structures.

Copyright © 2019 by ASME
Your Session has timed out. Please sign back in to continue.


Forum, T. C. , 2005, “ Chernobyl's Legacy: Health, Environmental and Socio-Economic Impacts,” International Atomic Energy Agency, Vienna, Austria, Report No. INIS-XA--903.
Rogers, E. , 1986, “ Report to the President by the Presidential Commission on the Space Shuttle Challenger Accident,” National Aeronautics and Space Administration, Washington, DC, Report No. AD-A171402.
Congress, U. , 2010, “ The Role of BP in the Deepwater Horizon Explosion and Oil Spill,” House of Representatives Subcommittee on Oversight and Investigations, Committee on Energy and Commerce, Washington, DC, Report No. 111-137.
Seife, C. , 2003, “ Columbia Disaster Underscores the Risky Nature of Risk Analysis,” Science, 299(5609), pp. 1001–1002. [CrossRef] [PubMed]
US Military Standard, 1980, “ Procedures for Performing a Failure Mode, Effect, and Criticality Analysis,” Department of Defense, Washington, DC, Standard No. MIL-STD-1629A.
Vesely, W. E. , Goldberg, F. F. , Roberts, N. H. , and Haasl, D. , 1981, Fault Tree Handbook (NUREG-0492), U.S. Nuclear Regulatory Commission, Washington, DC.
de Kleer, J. , and Kurien, J. , 2003, “ Fundamentals of Model-Based Diagnosis,” IFAC Proc. Vol., 36(5), pp. 25–36. [CrossRef]
Kurtoglu, T. , and Tumer, I. Y. , 2008, “ A Graph-Based Fault Identification and Propagation Framework for Functional Design of Complex Systems,” ASME J. Mech. Des., 130(5), p. 051401. [CrossRef]
Lawrence, E. , 2011, “ System Safety Analysis and Assessment for Part 23 Airplanes,” United States Federal Aviation Administration, Washington, DC, Report No. AC 25.1309-1A. https://www.faa.gov/regulations_policies/advisory_circulars/index.cfm/go/document.information/documentID/1019681
Wilkinson, P. J. , and Kelly, T. P. , 1998, “ Functional Hazard Analysis for Highly Integrated Aerospace Systems,” IEE Certification of Ground/Air Systems Seminar, London, Feb. 17, p. 4
SAE, 1996, “Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne System and Equipment,” Society of Automotive Engineers, Warrendale, PA, Standard No. ARP4761.
Ericson, C. A. , 2015, Hazard Analysis Techniques for System Safety, Wiley, Hoboken, NJ.
Delange, J. , Feiler, P. , Gluch, D. P. , and Hudak, J. , 2014, “ AADL Fault Modeling and Analysis Within an ARP4761 Safety Assessment,” Carnegie Mellon University Software Engineering Institute, Pittsburgh, PA, Report No. CMU/SEI-2014-TR-020 https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=311884.
Dowries, C. G. , and Chung, P. W. H. , 2011, “ Hazards in Advising Autonomy: Incorporating Hazard Modelling With System Dynamics Into the Aerospace Safety Assessment Process for UAS,” Sixth IET International Conference on System Safety, Birmingham, UK, Sept. 20–22, p. 12.
Joshi, A. , Heimdahl, M. , Miller, S. , and Whalen, M. , 2006, “ Model-Based Safety Analysis,” National Aeronautics and Space Administration, Washington, DC, Report No. NASA/CR-2006-213953.
Stone, R. B. , Tumer, I. Y. , and Van Wie, M. , 2004, “ The Function-Failure Design Method,” ASME J. Mech. Des., 127(3), pp. 397–407. [CrossRef]
Lough, K. G. , Stone, R. B. , and Tumer, I. , 2006, “ The Risk in Early Design (RED) Method: Likelihood and Consequence Formulations,” ASME Paper No. DETC2006-99375 .
Lough, K. G. , Stone, R. , and Tumer, I. Y. , 2009, “ The Risk in Early Design Method,” J. Eng. Des., 20(2), pp. 155–173. [CrossRef]
Hutcheson, R. S. , and Grantham, K. , 2012, “ Does Access to Expert Knowledge Allow Students to Better Assess Risk?,” ASME Paper No. DETC2012-71150.
Hollnagel, E. , 2017, FRAM: The Functional Resonance Analysis Method: Modelling Complex Socio-Technical Systems, CRC Press, Farnham, UK.
De Carvalho, P. V. R. , 2011, “ The Use of Functional Resonance Analysis Method (FRAM) in a Mid-Air Collision to Understand Some Characteristics of the Air Traffic Management System Resilience,” Reliab. Eng. Syst. Saf., 96(11), pp. 1482–1498. [CrossRef]
Rasmussen, B. , and Whetton, C. , 1997, “ Hazard Identification Based on Plant Functional Modelling,” Reliab. Eng. Syst. Saf., 55(2), pp. 77–84. [CrossRef]
Rasmussen, B. , Borch, K. , and Stärk, K. D. , 2001, “ Functional Modelling as Basis for Studying Individual and Organisational Factors–Application to Risk Analysis of Salmonella in Pork,” Food Control, 12(3), pp. 157–164. [CrossRef]
Papadopoulos, Y. , and McDermid, J. A. , 1999, “ Hierarchically Performed Hazard Origin and Propagation Studies,” International Conference on Computer Safety, Reliability, and Security, Toulouse, France, Sept. 27–29, pp. 139–152.
Nakao, H. , Katahira, M. , Miyamoto, Y. , and Leveson, N. , 2011, “ Safety Guided Design of Crew Return Vehicle in Concept Design Phase Using STAMP/STPA,” Fifth International Association for the Advancement of Space Safety Conference, Versailles, France, Oct. 17–19, pp. 497–501.
Laracy, J. R. , and Leveson, N. G. , 2007, “ Apply Stamp to Critical Infrastructure Protection,” IEEE Conference on Technologies for Homeland Security, Woburn, MA, May 16–17, pp. 215–220.
Dulac, N. , and Leveson, N. , 2004, “ An Approach to Design for Safety in Complex Systems,” International Symposium on Systems Engineering (INCOSE), pp. 517–530.
Ishimatsu, T. , Leveson, N. G. , Thomas, J. P. , Fleming, C. H. , Katahira, M. , Miyamoto, Y. , Ujiie, R. , Nakao, H. , and Hoshino, N. , 2014, “ Hazard Analysis of Complex Spacecraft Using Systems-Theoretic Process Analysis,” J. Spacecr. Rockets, 51(2), pp. 509–522. [CrossRef]
Jensen, D. , Tumer, I. Y. , and Kurtoglu, T. , 2009, “ Design of an Electrical Power System Using a Functional Failure and Flow State Logic Reasoning Methodology,” Prognostics and Health Management Society, pp. 1–13.
Coatanéa, E. , Nonsiri, S. , Ritola, T. , Tumer, I. Y. , and Jensen, D. C. , 2011, “ A Framework for Building Dimensionless Behavioral Models to Aid in Function-Based Failure Propagation Analysis,” ASME J. Mech. Des., 133(12), p. 121001. [CrossRef]
Papakonstantinou, N. , Sierla, S. , Jensen, D. C. , and Tumer, I. Y. , 2011, “ Capturing Interactions and Emergent Failure Behavior in Complex Engineered Systems at Multiple Scales,” ASME Paper No. DETC2011-47767.
Sierla, S. , Tumer, I. , Papakonstantinou, N. , Koskinen, K. , and Jensen, D. , 2012, “ Early Integration of Safety to the Mechatronic System Design Process by the Functional Failure Identification and Propagation Framework,” Mechatronics, 22(2), pp. 137–151. [CrossRef]
McIntire, M. G. , Keshavarzi, E. , Tumer, I. Y. , and Hoyle, C. , 2016, “ Functional Models With Inherent Behavior: Towards a Framework for Safety Analysis Early in the Design of Complex Systems,” ASME Paper No. IMECE2016-67040.
Li, Z. S. , and Mobin, M. S. , 2015, “ System Reliability Assessment Incorporating Interface and Function Failure,” IEEE Annual Reliability and Maintainability Symposium (RAMS), Palm Harbor, FL, Jan. 26–29, pp. 1–8.
Oh, Y. , Yoo, J. , Cha, S. , and Son, H. S. , 2005, “ Software Safety Analysis of Function Block Diagrams Using Fault Trees,” Reliab. Eng. Syst. Saf., 88(3), pp. 215–228. [CrossRef]
Meshkat, L. , Jenkins, S. , Mandutianu, S. , and Heron, V. , 2008, “ Automated Generation of Risk and Failure Models During Early Phase Design,” IEEE Aerospace Conference, Big Sky, MT, Mar. 1–8, pp. 1–12.
Krus, D. , and Lough, K. G. , 2009, “ Function-Based Failure Propagation for Conceptual Design,” Artif. Intell. Eng. Des. Anal. Manuf., 23(4), pp. 409–426. [CrossRef]
Keshavarzi, E. , McIntire, M. , Goebel, K. , Tumer, I. Y. , and Hoyle, C. , 2017, “ Resilient System Design Using Cost-Risk Analysis With Functional Models,” ASME Paper No. DETC2017-67952.
Keshavarzi, E. , 2018, “ Resilient Design for Complex Engineered Systems in the Early Design Phase.”
Short, A.-R. , Lai, A. D. , and Van Bossuyt, D. L. , 2018, “ Conceptual Design of Sacrificial Sub-Systems: Failure Flow Decision Functions,” Res. Eng. Des., 29(1), pp. 23–38.
Papadopoulos, Y. , Walker, M. , Parker, D. , Rüde, E. , Hamann, R. , Uhlig, A. , Grätz, U. , and Lien, R. , 2011, “ Engineering Failure Analysis and Design Optimisation With Hip-Hops,” Eng. Failure Anal., 18(2), pp. 590–608. [CrossRef]
Adachi, M. , Papadopoulos, Y. , Sharvia, S. , Parker, D. , and Tohdo, T. , 2011, “ An Approach to Optimization of Fault Tolerant Architectures Using Hip-Hops,” Software: Pract. Exper., 41(11), pp. 1303–1327. [CrossRef]
Mehr, A. F. , and Tumer, I. Y. , 2006, “ Risk-Based Decision-Making for Managing Resources During the Design of Complex Space Exploration Systems,” ASME J. Mech. Des., 128(4), pp. 1014–1022. [CrossRef]
Hoyle, C. , Tumer, I. Y. , Mehr, A. F. , and Chen, W. , 2009, “ Health Management Allocation During Conceptual System Design,” ASME J. Comput. Inf. Sci. Eng., 9(2), p. 021002. [CrossRef]
Pahl, G. , and Beitz, W. , 2007, Engineering Design: A Systematic Approach, Springer Science & Business Media, London.
Hulse, D. , Hoyle, C. , Goebel, K. , and Tumer, I. , 2018, “ Optimizing Function-Based Fault Propagation Model Resilience Using Expected Cost Scoring,” ASME Paper No. DETC2018-85318.
Erden, M. S. , Komoto, H. , van Beek, T. J. , D'Amelio, V. , Echavarria, E. , and Tomiyama, T. , 2008, “ A Review of Function Modeling: Approaches and Applications,” Artif. Intell. Eng. Des. Anal. Manuf., 22(2), pp. 147–169. [CrossRef]
Stone, R. B. , and Wood, K. L. , 2000, “ Development of a Functional Basis for Design,” ASME J. Mech. Des., 122(4), pp. 359–370. [CrossRef]
Kruse, B. , Gilz, T. , Shea, K. , and Eigner, M. , 2014, “ Systematic Comparison of Functional Models in SysML for Design Library Evaluation,” Proc. CIRP, 21, pp. 34–39. [CrossRef]
Ullman, D. , 2009, The Mechanical Design Process, McGraw-Hill Science/Engineering/Math, New York.
Ulrich, K. T. , and Eppinger, S. , 2012, Product Design and Development, McGraw-Hill Education, New York.
Wood, K. L. , Stone, R. B. , Mcadams, D. , Hirtz, J. , and Szykman, S. , 2002, “ A Functional Basis for Engineering Design: Reconciling and Evolving Previous Efforts,” National Institute of Standards and Technology, Washington, DC, Report No. 1447 http://citeseerx.ist.psu.edu/viewdoc/download?doi=
Jänsch, J. , and Birkhofer, H. , 2006, “ The Development of the Guideline VDI 2221-the Change of Direction,” DS 36: Ninth International Design Conference, Dubrovnik, Croatia (DESIGN 2006), pp. 45–52. https://www.designsociety.org/publication/18983/THE+DEVELOPMENT+OF+THE+GUIDELINE+VDI+2221+-+THE+CHANGE+OF+DIRECTION
Holling, C. S. , 1973, “ Resilience and Stability of Ecological Systems,” Annu. Rev. Ecol. Syst., 4(1), pp. 1–23. [CrossRef]
Holling, C. S. , 1996, “ Engineering Resilience Versus Ecological Resilience,” Engineering Within Ecological Constraints, National Academy, Washington, DC, pp. 31–44.
Pimm, S. L. , 1984, “ The Complexity and Stability of Ecosystems,” Nature, 307(5949), p. 321. [CrossRef]
Masten, A. S. , 2001, “ Ordinary Magic: Resilience Processes in Development,” Am. Psychol., 56(3), p. 227. [CrossRef] [PubMed]
Luthar, S. S. , Cicchetti, D. , and Becker, B. , 2000, “ The Construct of Resilience: A Critical Evaluation and Guidelines for Future Work,” Child Dev., 71(3), pp. 543–562. [CrossRef] [PubMed]
Briguglio, L. , Cordina, G. , Farrugia, N. , and Vella, S. , 2009, “ Economic Vulnerability and Resilience: Concepts and Measurements,” Oxford Dev. Stud., 37(3), pp. 229–247. [CrossRef]
Perrings, C. , 2006, “ Resilience and Sustainable Development,” Environ. Dev. Econ., 11(4), pp. 417–427. [CrossRef]
Saint-Arnaud, S. , and Bernard, P. , 2003, “ Convergence or Resilience? A Hierarchical Cluster Analysis of the Welfare Regimes in Advanced Countries,” Curr. Sociol., 51(5), pp. 499–527. [CrossRef]
Cohen, R. , Erez, K. , Ben-Avraham, D. , and Havlin, S. , 2000, “ Resilience of the Internet to Random Breakdowns,” Phys. Rev. Lett., 85(21), p. 4626. [CrossRef] [PubMed]
Ash, J. , and Newth, D. , 2007, “ Optimizing Complex Networks for Resilience Against Cascading Failure,” Physica A, 380, pp. 673–683. [CrossRef]
Sterbenz, J. P. , Hutchison, D. , Çetinkaya, E. K. , Jabbar, A. , Rohrer, J. P. , Schöller, M. , and Smith, P. , 2010, “ Resilience and Survivability in Communication Networks: Strategies, Principles, and Survey of Disciplines,” Comput. Networks, 54(8), pp. 1245–1265. [CrossRef]
Lengnick-Hall, C. A. , Beck, T. E. , and Lengnick-Hall, M. L. , 2011, “ Developing a Capacity for Organizational Resilience Through Strategic Human Resource Management,” Human Resour. Manage. Rev., 21(3), pp. 243–255. [CrossRef]
Ponomarov, S. Y. , and Holcomb, M. C. , 2009, “ Understanding the Concept of Supply Chain Resilience,” Int. J. Logist. Manage., 20(1), pp. 124–143. [CrossRef]
Chen, X. , Xi, Z. , and Jing, P. , 2017, “ A Unified Framework for Evaluating Supply Chain Reliability and Resilience,” IEEE Trans. Reliab., 66(4), pp. 1144–1156. [CrossRef]
Linkov, I. , Bridges, T. , Creutzig, F. , Decker, J. , Fox-Lent, C. , Kröger, W. , Lambert, J. H. , Levermann, A. , Montreuil, B. , Nathwani, J. , Nyer, R. , Renn, O. , Scharte, B. , Scheffler, A. , Schreurs, M. , and Thiel-Clemen, T. , 2014, “ Changing the Resilience Paradigm,” Nat. Clim. Change, 4(6), p. 407. [CrossRef]
Hosseini, S. , Barker, K. , and Ramirez-Marquez, J. E. , 2016, “ A Review of Definitions and Measures of System Resilience,” Reliab. Eng. Syst. Saf., 145, pp. 47–61. [CrossRef]
Yodo, N. , and Wang, P. , 2016, “ Engineering Resilience Quantification and System Design Implications: A Literature Survey,” ASME J. Mech. Des., 138(11), p. 111408. [CrossRef]
Li, J. , and Xi, Z. , 2014, “ Engineering Recoverability: A New Indicator of Design for Engineering Resilience,” ASME Paper No. DETC2014-35005.
Hazelrigg, G. A. , 1998, “ A Framework for Decision-Based Engineering Design,” ASME J. Mech. Des., 120(4), pp. 653–658. [CrossRef]
Von Neumann, J. , and Morgenstern, O. , 2007, Theory of Games and Economic Behavior (Commemorative Edition), Princeton University Press, Princeton, NJ.
Thurston, D. L. , 2006, “ Utility Function Fundamentals,” Decision Making in Engineering Design, ASME Press, New York.
Hazelrigg, G. A. , 1999, “ An Axiomatic Framework for Engineering Design,” ASME J. Mech. Des., 121(3), pp. 342–347. [CrossRef]
Gu, X. , Renaud, J. E. , Ashe, L. M. , Batill, S. M. , Budhiraja, A. S. , and Krajewski, L. J. , 2002, “ Decision-Based Collaborative Optimization,” ASME J. Mech. Des., 124(1), pp. 1–13. [CrossRef]
Wassenaar, H. J. , and Chen, W. , 2003, “ An Approach to Decision-Based Design With Discrete Choice Analysis for Demand Modeling,” ASME J. Mech. Des., 125(3), pp. 490–497. [CrossRef]
Collopy, P. D. , and Hollingsworth, P. M. , 2011, “ Value-Driven Design,” J. Aircr., 48(3), pp. 749–759. [CrossRef]
Kmenta, S. , and Ishii, K. , 2000, “ Scenario-Based FMEA: A Life Cycle Cost Perspective,” ASME Paper No. DETC2000/RSAFP-14478.
Hu, C. , and MacKenzie, C. A. , 2017, “ Optimizing Resilience When Designing Engineered Systems,” ASME Paper No. DETC2017-68387.
Haimes, Y. Y. , 2009, “ On the Definition of Resilience in Systems,” Risk Anal., 29(4), pp. 498–501. [CrossRef] [PubMed]
Henry, D. , and Ramirez-Marquez, J. E. , 2012, “ Generic Metrics and Quantitative Approaches for System Resilience as a Function of Time,” Reliab. Eng. Syst. Saf., 99, pp. 114–122. [CrossRef]
Hulse, D. , Tumer, K. , Hoyle, C. , and Tumer, I. , 2018, “ Modeling Multidisciplinary Design With Multiagent Learning,” Artif. Intell. Eng. Des. Anal. Manuf. (epub).
Helms, B. , Shea, K. , and Hoisl, F. , 2009, “ A Framework for Computational Design Synthesis Based on Graph-Grammars and Function-Behavior-Structure,” ASME Paper No. DETC2009-86851 .
Sridharan, P. , and Campbell, M. I. , 2004, “ A Grammar for Function Structures,” ASME Paper No. DETC2004-57130 .
Martins, J. R. , and Lambe, A. B. , 2013, “ Multidisciplinary Design Optimization: A Survey of Architectures,” AIAA J., 51(9), pp. 2049–2075. [CrossRef]
Vesely, W. , Fragola, J. , Minarick, J. , and Railsback, Ja. , 2002, “ Fault Tree Handbook With Aerospace Applications,” NASA Office of Safety and Mission Assurance, Washington, DC, accessed Oct. 16, 2018, https://elibrary.gsfc.nasa.gov/_assets/doclibBidder/tech_docs/25.%20NASA_Fault_Tree_Handbook_with_Aerospace_Applications%20-%20Copy.pdf
Keshavarzi, E. , McIntire, M. , and Hoyle, C. , 2015, “ Dynamic Design Using the Kalman Filter for Flexible Systems With Epistemic Uncertainty,” ASME Paper No. DETC2015-46378.


Grahic Jump Location
Fig. 1

Illustration of a fault propagation simulation using IBFM. A fault propagates from an initiating mode through the flows of the functional model until it produces an end-state with resulting fault modes and flow health states.

Grahic Jump Location
Fig. 2

Costs associated with a failure event in a resilient system

Grahic Jump Location
Fig. 3

Illustration of fault re-simulation required to capture the costs of partial recovery Cr

Grahic Jump Location
Fig. 4

Functional model of a signal-carrying medium, with modes, conditions, costs, and probabilities associated with each function

Grahic Jump Location
Fig. 5

Framework enabled by integrating cost-based scoring and fault simulation. The designer sets up a parameterized design problem which is then solved by an optimization algorithm.

Grahic Jump Location
Fig. 6

Functional model of base monopropellant system

Grahic Jump Location
Fig. 7

Example controlling function conditions and modes

Grahic Jump Location
Fig. 8

Cost optimization of the functional model using the evolutionary algorithm, showing how value can be increased using the presented optimization framework

Grahic Jump Location
Fig. 9

Differential costs of design variants based on fault simulation

Grahic Jump Location
Fig. 10

Design variant 1: redundant gas tanks

Grahic Jump Location
Fig. 11

Design variant 2: redundant thrusters

Grahic Jump Location
Fig. 12

Design variant 3: heat recovery system

Grahic Jump Location
Fig. 13

Design variant 4: redundant pressure regulators

Grahic Jump Location
Fig. 14

Design variant 5: optimized control features



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In