0
Research Papers

Characterizing Uncertainty Attributable to Surrogate Models

[+] Author and Article Information
Jie Zhang

Postdoctoral Research Associate
Mem. ASME
Multidisciplinary Design and Optimization
Laboratory (MDOL),
Department of Mechanical and
Aerospace Engineering,
Syracuse University,
Syracuse, NY 13244
e-mail: jzhang56@syr.edu

Souma Chowdhury

Assistant Research Professor
Mem. ASME
Department of Mechanical Engineering,
Department of Electrical and
Computer Engineering,
Mississippi State University,
Mississippi State, MS 39762
e-mail: souma.chowdhury@msstate.edu

Ali Mehmani

Multidisciplinary Design and Optimization
Laboratory (MDOL),
Department of Mechanical and
Aerospace Engineering,
Syracuse University,
Syracuse, NY 13244
e-mail: amehmani@syr.edu

Achille Messac

Dean
Professor
Earnest W. and Mary Ann Deavenport Jr.
Endowed Chair,
Fellow ASME
Department of Aerospace Engineering,
Bagley College of Engineering,
Mississippi State University,
Mississippi State, MS 39762
e-mail: messac@bagley.msstate.edu

1Present address: National Renewable Energy Laboratory (NREL), Golden, CO 80401.

2Corresponding author.

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received December 13, 2012; final manuscript received October 23, 2013; published online January 10, 2014. Assoc. Editor: Irem Y. Tumer.

J. Mech. Des 136(3), 031004 (Jan 10, 2014) (11 pages) Paper No: MD-12-1608; doi: 10.1115/1.4026150 History: Received December 13, 2012; Revised October 23, 2013

This paper investigates the characterization of the uncertainty in the prediction of surrogate models. In the practice of engineering, where predictive models are pervasively used, the knowledge of the level of modeling error in any region of the design space is uniquely helpful for design exploration and model improvement. The lack of methods that can explore the spatial variation of surrogate error levels in a wide variety of surrogates (i.e., model-independent methods) leaves an important gap in our ability to perform design domain exploration. We develop a novel framework, called domain segmentation based on uncertainty in the surrogate (DSUS) to segregate the design domain based on the level of local errors. The errors in the surrogate estimation are classified into physically meaningful classes based on the user's understanding of the system and/or the accuracy requirements for the concerned system analysis. The leave-one-out cross-validation technique is used to quantity the local errors. Support vector machine (SVM) is implemented to determine the boundaries between error classes, and to classify any new design point into the pertinent error class. We also investigate the effectiveness of the leave-one-out cross-validation technique in providing a local error measure, through comparison with actual local errors. The utility of the DSUS framework is illustrated using two different surrogate modeling methods: (i) the Kriging method and (ii) the adaptive hybrid functions (AHF). The DSUS framework is applied to a series of standard test problems and engineering problems. In these case studies, the DSUS framework is observed to provide reasonable accuracy in classifying the design-space based on error levels. More than 90% of the test points are accurately classified into the appropriate error classes.

FIGURES IN THIS ARTICLE
<>
Copyright © 2014 by ASME
Your Session has timed out. Please sign back in to continue.

References

Figures

Grahic Jump Location
Fig. 1

The illustration of the prediction uncertainty modeling (class 1: low error; class 2: medium error; class 3: high error)

Grahic Jump Location
Fig. 2

The framework of the DSUS methodology

Grahic Jump Location
Fig. 3

Comparison of cross-validation errors and actual local errors (1-variable function): (a) AHF and (b) Kriging

Grahic Jump Location
Fig. 4

Comparison of cross-validation errors and actual local errors (Dixon & Price function): (a) actual local errors (AHF), (b) cross-validation errors (AHF), (c) actual local errors (Kriging), and (d) cross-validation errors (Kriging)

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In