Research Papers: Design Automation

Reliability-Based Design Optimization Using Confidence-Based Model Validation for Insufficient Experimental Data

[+] Author and Article Information
Min-Yeong Moon

Department of Mechanical and
Industrial Engineering,
College of Engineering,
The University of Iowa,
Iowa City, IA 52242
e-mail: minyeong-moon@uiowa.edu

K. K. Choi

Department of Mechanical and
Industrial Engineering,
College of Engineering,
The University of Iowa,
Iowa City, IA 52242
e-mail: kyung-choi@uiowa.edu

Hyunkyoo Cho

Department of Mechanical and
Industrial Engineering,
College of Engineering,
The University of Iowa,
Iowa City, IA 52242
e-mail: hyunkyoo-cho@uiowa.edu

Nicholas Gaul

RAMDO Solutions, LLC,
Iowa City, IA 52240
e-mail: nicholas-gaul@ramdosolution.com

David Lamb

Warren, MI 48397-5000
e-mail: david.lamb@us.army.mil

David Gorsich

Warren, MI 48397-5000
e-mail: david.j.gorsich.civ@mail.mil

1Corresponding author.

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received August 6, 2016; final manuscript received December 22, 2016; published online January 27, 2017. Assoc. Editor: Nam H. Kim.This work is in part a work of the U.S. Government. ASME disclaims all interest in the U.S. Government's contributions

J. Mech. Des 139(3), 031404 (Jan 27, 2017) (10 pages) Paper No: MD-16-1560; doi: 10.1115/1.4035679 History: Received August 06, 2016; Revised December 22, 2016

The conventional reliability-based design optimization (RBDO) methods assume that a simulation model is able to represent the real physics accurately. However, this assumption may not always hold as the simulation model could be biased. Accordingly, designed product based on the conventional RBDO optimum may either not satisfy the target reliability or be overly conservative design. Therefore, simulation model validation using output experimental data, which corrects model bias, should be integrated in the RBDO process. With particular focus on RBDO, the model validation needs to account for the uncertainty induced by insufficient experimental data as well as the inherent variability of the products. In this paper, a confidence-based model validation method that captures the variability and the uncertainty, and that corrects model bias at a user-specified target confidence level, has been developed. The developed model validation helps RBDO to obtain a conservative RBDO optimum design at the target confidence level. The RBDO with model validation may have a convergence issue because the feasible domain changes as the design moves (i.e., a moving-target problem). To resolve the issue, a practical optimization procedure is proposed. Furthermore, the efficiency is achieved by carrying out deterministic design optimization (DDO) and RBDO without model validation, followed by RBDO with confidence-based model validation. Finally, we demonstrate that the proposed RBDO approach can achieve a conservative and practical optimum design given a limited number of experimental data.

Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.


Klir, G. J. , and Folger, T. A. , 1988, Fuzzy Sets, Uncertainty, and Information, Prentice Hall, Englewood Cliffs, NJ.
Oberkampf, W. L. , Helton, J. C. , and Sentz, K. , 2001, “ Mathematical Representation of Uncertainty,” AIAA Paper No. 2001-1645.
Oberkampf, W. L. , and Roy, C. J. , 2010, Verification and Validation in Scientific Computing, Cambridge University Press, New York.
Lee, I. , Choi, K. K. , and Zhao, L. , 2011, “ Sampling-Based RBDO Using the Dynamic Kriging Method and Stochastic Sensitivity Analysis,” Struct. Multidiscip. Optim., 44(3), pp. 299–317. [CrossRef]
Lee, I. , Choi, K. K. , Noh, Y. , Zhao, L. , and Gorsich, D. , 2011, “ Sampling-Based Stochastic Sensitivity Analysis Using Score Functions for RBDO Problems With Correlated Random Variables,” ASME J. Mech. Des., 133(2), p. 021003. [CrossRef]
Hasofer, A. M. , and Lind, N. C. , 1974, “ An Exact and Invariant First Order Reliability Format,” J. Eng. Mech., 100(1), pp. 111–121.
Mahadevan, S. , and Haldar, A. , 2000, Probability, Reliability and Statistical Methods in Engineering Design, Wiley, New York.
Chiralaksanakul, A. , and Mahadevan, S. , 2005, “ First-Order Approximation Methods in Reliability-Based Design Optimization,” ASME J. Mech. Des., 127(5), pp. 851–857. [CrossRef]
Tu, J. , Choi, K. K. , and Park, Y. H. , 1999, “ A New Study on Reliability-Based Design Optimization,” ASME J. Mech. Des., 121(4), pp. 557–564. [CrossRef]
Hohenbichler, M. , and Rackwitz, R. , 1988, “ Improvement of Second-Order Reliability Estimates by Importance Sampling,” ASCE J. Eng. Mech., 114(12), pp. 2195–2199. [CrossRef]
Kennedy, M. C. , and O'Hagan, A. , 2001, “ Bayesian Calibration of Computer Models,” J. R. Stat. Soc.: Ser. B (Stat. Methodol.), 63(3), pp. 425–464. [CrossRef]
Oberkampf, W. L. , and Barone, M. F. , 2006, “ Measures of Agreement Between Computation and Experiment: Validation Metrics,” J. Comput. Phys., 217(1), pp. 5–36. [CrossRef]
Chen, W. , Xiong, Y. , Tsui, K. L. , and Wang, S. , 2006, “ Some Metrics and a Bayesian Procedure for Validating Predictive Models in Engineering Design,” ASME Paper No. DETC2006-99599.
Ferson, S. , Oberkampf, W. L. , and Ginzburg, L. , 2008, “ Model Validation and Predictive Capability for the Thermal Challenge Problem,” Comput. Methods Appl. Mech. Eng., 197(29), pp. 2408–2430. [CrossRef]
Loeppky, J. , Bingham, D. , and Welch, W. , 2006, “ Computer Model Calibration or Tuning in Practice,” University of British Columbia, Vancouver, BC, Canada, Report No. 221.
Xiong, Y. , Chen, W. , Tsui, K. L. , and Apley, D. , 2009, “ A Better Understanding of Model Updating Strategies in Validating Engineering Models,” Comput. Methods Appl. Mech. Eng., 198(15–16), pp. 1327–1337. [CrossRef]
Youn, B. D. , Jung, B. C. , Xi, Z. , Kim, S. B. , and Lee, W. R. , 2011, “ A Hierarchical Framework for Statistical Model Calibration in Engineering Product Development,” Comput. Methods Appl. Mech. Eng., 200(13–16), pp. 1421–1431. [CrossRef]
Drignei, D. , Mourelatos, Z. P. , Kokkolaras, M. , Pandey, V. , and Koscik, G. , 2012, “ A Variable-Size Local Domain Approach for Increased Design Confidence in Simulation-Based Optimization,” Struct. Multidiscip. Optim., 46(1), pp. 83–92. [CrossRef]
Drignei, D. , Mourelatos, Z. P. , Pandey, V. , and Kokkolaras, M. , 2012, “ Concurrent Design Optimization and Calibration-Based Validation using Local Domains Sized by Bootstrapping,” ASME J. Mech. Des., 134(10), p.100910. [CrossRef]
Xi, Z. , Fu, Y. , and Yang, R. J. , 2013, “ Model Bias Characterization in the Design Space Under Uncertainty,” Int. J. Performability Eng., 9(4), pp. 433–444.
Jiang, Z. , Chen, W. , Fu, Y. , and Yang, R. J. , 2013, “ Reliability-Based Design Optimization With Model Bias and Data Uncertainty,” SAE Int. J. Mater. Manuf., 6(3), pp. 502–516. [CrossRef]
Xi, Z. , Hao, P. , Fu, Y. , and Yang, R. J. , 2014, “ A Copula-Based Approach for Model Bias Characterization,” SAE Int. J. Passenger Cars-Mech. Syst., 7(2), pp. 781–786. [CrossRef]
Hao, P. , Xi, Z. , and Yang, R. J. , 2016, “ Model Uncertainty Approximation Using a Copula-Based Approach for Reliability Based Design Optimization,” Struct. Multidiscip. Optim., 54, pp. 1–14. [CrossRef]
Higdon, D. , Nakhleh, C. , Gattiker, J. , and Williams, B. , 2008, “ A Bayesian Calibration Approach to the Thermal Problem,” Comput. Methods Appl. Mech. Eng., 197(29–32), pp. 2431–2441. [CrossRef]
Arendt, P. , Apley, D. , and Chen, W. , 2012, “ Quantification of Model Uncertainty: Calibration, Model Discrepancy, and Identifiability,” ASME J. Mech. Des., 134(10), p. 100908. [CrossRef]
Arendt, P. , Apley, D. , Chen, W. , Lamb, D. , and Gorsich, D. , 2012, “ Improving Identifiability in Model Calibration Using Multiple Responses,” ASME J. Mech. Des., 134(10), p. 100909. [CrossRef]
Picheny, V. , Kim, N. H. , and Haftka, R. T. , 2010, “ Application of Bootstrap Method in Conservative Estimation of Reliability With Limited Samples,” Struct. Multidiscip. Optim., 41(2), pp. 205–217. [CrossRef]
Wang, Z. , and Wang, P. , 2014, “ A Maximum Confidence Enhancement Based Sequential Sampling Scheme for Simulation-Based Design,” ASME J. Mech. Des., 136(2), p. 021006. [CrossRef]
Park, C. , and Kim, N. H. , 2016, “ Safety Envelope for Load Tolerance of Structural Element Design Based on Multi-Stage Testing,” Adv. Mech. Eng., 8(9), pp. 1–11. [CrossRef]
Gunawan, S. , and Papalambros, P. Y. , 2006, “ A Bayesian Approach to Reliability-Based Optimization With Incomplete Information,” ASME J. Mech. Des., 128(4), pp. 909–918. [CrossRef]
Youn, B. D. , and Wang, P. , 2008, “ Bayesian Reliability-Based Design Optimization Using Eigenvector Dimension Reduction (EDR) Method,” Struct. Multidiscip. Optim., 36(2), pp. 107–123. [CrossRef]
Choi, J. , An, D. , and Won, J. , 2010, “ Bayesian Approach for Structural Reliability Analysis and Optimization Using the Kriging Dimension Reduction Method,” ASME J. Mech. Des., 132(5), p. 051003. [CrossRef]
Noh, Y. , Choi, K. K. , Lee, I. , Gorsich, D. , and Lamb, D. , 2011, “ Reliability-Based Design Optimization With Confidence Level Under Input Model Uncertainty Due to Limited Test Data,” Struct. Multidiscip. Optim., 43(4), pp. 443–458. [CrossRef]
Noh, Y. , Choi, K. K. , Lee, I. , and Gorsich, D. , 2011, “ Reliability-Based Design Optimization With Confidence Level for Non-Gaussian Distributions Using Bootstrap Method,” ASME J. Mech. Des., 133(9), p. 091001. [CrossRef]
Cho, H. , Choi, K. K. , Gaul, N. , Lee, I. , Lamb, D. , and Gorsich, D. , 2016, “ Conservative Reliability-Based Design Optimization Method With Insufficient Input Data,” Struct. Multidiscip. Optim., 54(6), pp. 1–22.
Zaman, K. , and Mahadevan, S. , 2016, “ Reliability-Based Design Optimization of Multidisciplinary System Under Aleatory and Epistemic Uncertainty,” Struct. Multidiscip. Optim., 2016, pp. 1–19.
Silverman, B. W. , 1986, Density Estimation for Statistics and Data Analysis, Chapman & Hall, London.
Moon, M. Y. , Choi, K. K. , Cho, H. , Gaul, N. , Lamb, D. , and Gorsich, D. , 2015, “ Development of a Conservative Model Validation Approach for Reliable Analysis,” ASME Paper No. DETC2015-46982.
Martins, J. R. , Sturdza, P. , and Alonso, J. J. , 2003, “ The Complex-Step Derivative Approximation,” ACM Trans. Math. Software (TOMS), 29(3), pp. 245–262. [CrossRef]
Zhao, L. , Choi, K. K. , and Lee, I. , 2011, “ Metamodeling Method Using Dynamic Kriging for Design Optimization,” AIAA J., 49(9), pp. 2034–2046. [CrossRef]
Song, H. , Choi, K. K. , and Lamb, D. , 2013, “ A Study on Improving the Accuracy of Kriging Models by Using Correlation Model/Mean Structure Selection and Penalized Log-Likelihood Function,” 10th World Congress on Structural and Multidisciplinary Optimization, Orlando, FL, May 19–24.
Volpi, S. , Diez, M. , Gaul, N. J. , Song, H. , Iemma, U. , Choi, K. K. , Campana, E. F. , and Stern, F. , 2014, “ Development and Validation of a Dynamic Metamodel Based on Stochastic Radial Basis Functions and Uncertainty Quantification,” Struct. Multidiscip. Optim., 51(2), pp. 1–22.
Sen, O. , Davis, S. , Jacobs, G. , and Udaykumar, H. S. , 2015, “ Evaluation of Convergence Behavior of Metamodeling Techniques for Bridging Scales in Multi-Scale Multimaterial Simulation,” J. Comput. Phys., 294, pp. 585–604. [CrossRef]


Grahic Jump Location
Fig. 1

Illustration of output PDFs

Grahic Jump Location
Fig. 2

Contour of cost function and limit states of biased simulation output G(x) and true output Gtrue(x)

Grahic Jump Location
Fig. 3

Histogram of confidence-based probabilities of failure among 1000 tests: (a) Constraint G1 and (b) constraint G2

Grahic Jump Location
Fig. 4

RBDO optimum using confidence-based model validation for insufficient experimental data



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In