Research Papers: Design Automation

Validating Dynamic Engineering Models Under Uncertainty

[+] Author and Article Information
Zequn Wang

Department of Mechanical Engineering,
Northwestern University,
Evanston, IL 60208
e-mail: zequn.wang@northwestern.edu

Yan Fu

Research and Advanced Engineering,
Ford Motor Company,
Dearborn, MI 48121
e-mail: yfu4@ford.com

Ren-Jye Yang

Research and Advanced Engineering,
Ford Motor Company,
Dearborn, MI 48121
e-mail: ryang@ford.com

Saeed Barbat

Research and Advanced Engineering,
Ford Motor Company,
Dearborn, MI 48121
e-mail: sbarbat@ford.com

Wei Chen

Department of Mechanical Engineering,
Northwestern University,
Evanston, IL 60208
e-mail: weichen@northwestern.edu

1Corresponding author.

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received February 16, 2016; final manuscript received May 30, 2016; published online September 12, 2016. Assoc. Editor: Zissimos P. Mourelatos.

J. Mech. Des 138(11), 111402 (Sep 12, 2016) (12 pages) Paper No: MD-16-1128; doi: 10.1115/1.4034089 History: Received February 16, 2016; Revised May 30, 2016

Validating dynamic engineering models is critically important in practical applications by assessing the agreement between simulation results and experimental observations. Though significant progresses have been made, the existing metrics lack the capability of managing uncertainty in both simulations and experiments. In addition, it is challenging to validate a dynamic model aggregately over both the time domain and a model input space with data at multiple validation sites. To overcome these difficulties, this paper presents an area-based metric to systemically handle uncertainty and validate computational models for dynamic systems over an input space by simultaneously integrating the information from multiple validation sites. To manage the complexity associated with a high-dimensional data space, eigenanalysis is performed for the time series data from simulations at each validation site to extract the important features. A truncated Karhunen–Loève (KL) expansion is then constructed to represent the responses of dynamic systems, resulting in a set of uncorrelated random coefficients with unit variance. With the development of a hierarchical data-fusion strategy, probability integral transform (PIT) is then employed to pool all the resulting random coefficients from multiple validation sites across the input space into a single aggregated metric. The dynamic model is thus validated by calculating the cumulative area difference of the cumulative density functions. The proposed model validation metric for dynamic systems is illustrated with a mathematical example, a supported beam problem with stochastic loads, and real data from the vehicle occupant-restraint system.

Copyright © 2016 by ASME
Your Session has timed out. Please sign back in to continue.


Thacker, B. H. , Doebling, S. W. , Hemez, F. M. , Anderson, M. C. , Pepin, J. E. , and Rodriguez, E. A. , 2004, “ Concepts of Model Verification and Validation,” National Laboratory, Los Alamos, NM, Report No. LA-14167.
DOD, 1996, “ Verification, Validation, and Accreditation (VV&A) Recommended Practices Guide,” Department of Defense, Alexandria, VA.
DOE, 2000, “ Accelerated Strategic Computing Initiative (ASCI) Program Plan,” Department of Energy, Washington, DC, Report No. DOE/DP-99-000010592.
AIAA, 1998, “ Guide for the Verification and Validation of Computational Fluid Dynamics Simulations,” American Institute of Aeronautics and Astronautics, Reston, VA, Standard No. AIAA-G-077-1998.
ASME, 2006, “ Guide for Verification and Validation in Computational Solid Mechanics,” American Society of Mechanical Engineers, New York, Standard No. ASME V V 10-2006.
Chen, W. , Baghdasaryan, L. , Buranathiti, T. , and Cao, J. , 2004, “ Model Validation Via Uncertainty Propagation and Data Transformation,” AIAA J., 42(7), pp. 1406–1415. [CrossRef]
Mahadevan, S. , and Rebba, R. , 2005, “ Validation of Reliability Computational Models Using Bayes Networks,” Reliab. Eng. Syst. Saf., 87(2), pp. 223–232. [CrossRef]
Liu, Y. , Chen, W. , Arendt, P. , and Huang, H. Z. , 2011, “ Toward a Better Understanding of Model Validation Metrics,” ASME J. Mech. Des., 133(7), p. 071005. [CrossRef]
Oberkampf, W. L. , Trucano, T. G. , and Hirsch, C. , 2004, “ Verification, Validation, and Predictive Capability in Computational Engineering and Physics,” ASME Appl. Mech. Rev., 57(5), pp. 345–384. [CrossRef]
Kennedy, M. C. , and O'Hagan, A. , 2001, “ Bayesian Calibration of Computer Models,” J R. Stat. Soc. Ser. B, Stat. Methodol., 63(3), pp. 425–464. [CrossRef]
Mayer, D. G. , and Butler, D. G. , 1993 “ Statistical Validation,” Ecol. Modell., 68(1–2), pp. 21–32. [CrossRef]
Hills, R. G. , and Trucano, T. G. , 1999, “ Statistical Validation of Engineering and Scientific Models: Background,” Sandia National Laboratories, Albuquerque, NM, Report No. SAND 99-1256.
Sugawara, Y. , Shinohara, K. , and Kobayashi, N. , 2009, “ Quantitative Validation of Dynamic Stiffening Represented by Absolute Nodal Coordinate Formulation,” ASME Paper No. DETC2009-86955.
Marden, J. , 2000, “ Hypothesis Testing: From p Values to Bayes Factors,” J. Am. Stat. Assoc., 95(452), pp. 1316–1320.
O'Hagan, A. , 1995, “ Fractional Bayes Factors for Model Comparison,” J. R. Stat. Soc. Ser. B, 57(1), pp. 99–138.
Kass, R. , and Raftery, A. , 1995, “ Bayes Factors,” J. Am. Stat. Assoc., 90(430), pp. 773–795. [CrossRef]
Srivastava, M. S. , 2002, Methods of Multivariate Statistics, 1st ed., Wiley, New York.
Oberkampf, W. , and Barone, M. , 2006, “ Measures of Agreement Between Computation and Experiment: Validation Metrics,” J. Comput. Phys., 217(1), pp. 5–36. [CrossRef]
Rebba, R. , and Mahadevan, S. , 2008, “ Computational Methods for Model Reliability Assessment,” Reliab. Eng. Syst. Saf., 93(8), pp. 1197–1207. [CrossRef]
Rebba, R. , Mahadevan, S. , and Huang, S. , 2006, “ Validation and Error Estimation of Computational Models,” Reliab. Eng. Syst. Saf., 91(10) pp. 1390–1397. [CrossRef]
Ferson, S. , Oberkampf, W. , and Ginzburg, L. , 2008, “ Model Validation and Predictive Capability for the Thermal Challenge Problem,” Comput. Methods Appl. Mech. Eng., 197(29–32), pp. 2408–2430. [CrossRef]
Ferson, S. , and Oberkampf, W. , 2009, “ Validation of Imprecise Probability Models,” Int. J. Reliab. Saf., 3(1), pp. 3–22. [CrossRef]
Sarin, H. , Kokkolaras, M. , Hulbert, G. , Papalambros, P. , Barbat, S. , and Yang, R. J. , 2010, “ Comparing Time Histories for Validation of Simulation Models: Error Measures and Metrics,” ASME J. Dyn. Syst. Meas. Control, 132(6), p. 0614011. [CrossRef]
Gehre, C. , Gades, H. , and Wernicke, P. , 2009, “ Objective Rating of Signals Using Test and Simulation Responses,” 21st ESV Conference, Stuttgart, Germany, Paper No. 09-0407.
Zhan, Z. , Fu, Y. , and Yang, R.-J. , 2011, “ Enhanced Error Assessment of Response Time Histories (EEARTH) Metric and Calibration Process,” SAE 2011 World Congress, Detroit, MI, SAE Paper No. 2011-01-0245.
Jiang, X. , and Mahadevan, S. , 2008, “ Bayesian Wavelet Method for Multivariate Model Assessment of Dynamical Systems,” J. Sound Vib., 312(4–5), pp. 694–712. [CrossRef]
Jiang, X. , and Mahadevan, S. , 2011, “ Wavelet Spectrum Analysis Approach to Model Validation of Dynamic Systems,” Mech. Syst. Signal Process., 25(2), pp. 575–590. [CrossRef]
Zhan, Z. , Fu, Y. , Yang, R.-J. , and Peng, Y. , 2011, “ An Enhanced Bayesian Based Model Validation Method for Dynamic Systems,” ASME J. Mech. Des., 133(4), p. 041005. [CrossRef]
Xi, Z. , Pan, H. , Fu, Y. , and Yang, R. , 2015, “ Validation Metric for Dynamic System Responses Under Uncertainty,” SAE Int. J. Mater. Manuf., 8(2), pp. 309–314. [CrossRef]
Huang, S. P. , Quek, S. T. , and Phoon, K. K. , 2001, “ Convergence Study of the Truncated Karhunen–Loeve Expansion for Simulation of Stochastic Processes,” Int. J. Numer. Methods Eng., 52(9), pp. 1029–1043. [CrossRef]
Phoon, K. K. , Huang, S. P. , and Quek, S. T. , 2002, “ Simulation of Second-Order Processes Using Karhunen–Loeve Expansion,” Comput. Struct., 80(12), pp. 1049–1060. [CrossRef]
Phoon, K. K. , Huang, H. W. , and Quek, S. T. , 2005, “ Simulation of Strongly Non-Gaussian Processes Using Karhunen–Loève Expansion,” Probab. Eng. Mech., 20(2), pp. 188–198. [CrossRef]


Grahic Jump Location
Fig. 1

Model validation of dynamic systems

Grahic Jump Location
Fig. 2

Variation-Based KL transformation

Grahic Jump Location
Fig. 3

Reconstructions of a random process using the truncated KL expansion

Grahic Jump Location
Fig. 4

Hierarchical data structure

Grahic Jump Location
Fig. 5

PIT for the ith coefficient

Grahic Jump Location
Fig. 6

Area-based validation metric

Grahic Jump Location
Fig. 7

Mean function of the random responses

Grahic Jump Location
Fig. 8

Principal eigenvalues in spectral decomposition

Grahic Jump Location
Fig. 9

Empirical probability density function of the coefficients and area-based dynamic model validation metric

Grahic Jump Location
Fig. 10

Area-based model validation metric

Grahic Jump Location
Fig. 11

Observations and results for the first case

Grahic Jump Location
Fig. 12

Validation metric for cases 2–4

Grahic Jump Location
Fig. 13

Geometry of the supported beam

Grahic Jump Location
Fig. 14

Finite-element analysis of the supported beam

Grahic Jump Location
Fig. 15

Experimental observations of the supported beam

Grahic Jump Location
Fig. 16

Coefficients C1 and C2 for the first validation site

Grahic Jump Location
Fig. 17

Area-based validation metric

Grahic Jump Location
Fig. 18

Chest acceleration in the X-direction: models versus tests

Grahic Jump Location
Fig. 19

Ten significant eigenvalues

Grahic Jump Location
Fig. 20

Area-based validation metric



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In