Research Papers: Design Automation

An Advanced and Robust Ensemble Surrogate Model: Extended Adaptive Hybrid Functions

[+] Author and Article Information
Xueguan Song

School of Mechanical Engineering,
Dalian University of Technology,
No. 2 Linggong Road,
Ganjingzi District,
Dalian 116024, China
e-mail: sxg@dlut.edu.cn

Liye Lv

School of Mechanical Engineering,
Dalian University of Technology,
No. 2 Linggong Road,
Ganjingzi District,
Dalian 116024, China
e-mail: lvhexiaoye@mail.dlut.edu.cn

Jieling Li

School of Mechanical Engineering,
Dalian University of Technology,
No. 2 Linggong Road,
Ganjingzi District,
Dalian 116024, China
e-mail: 349783872@mail.dlut.edu.cn

Wei Sun

School of Mechanical Engineering,
Dalian University of Technology,
No. 2 Linggong Road,
Ganjingzi District,
Dalian 116024, China
e-mail: sunwei@dlut.edu.cn

Jie Zhang

Department of Mechanical Engineering,
The University of Texas at Dallas,
Richardson, TX 75080
e-mail: jiezhang@utdallas.edu

1Corresponding author.

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received February 19, 2017; final manuscript received January 5, 2018; published online February 27, 2018. Assoc. Editor: Christina Bloebaum.

J. Mech. Des 140(4), 041402 (Feb 27, 2018) (9 pages) Paper No: MD-17-1144; doi: 10.1115/1.4039128 History: Received February 19, 2017; Revised January 05, 2018

Hybrid or ensemble surrogate models developed in recent years have shown a better accuracy compared to individual surrogate models. However, it is still challenging for hybrid surrogate models to always meet the accuracy, robustness, and efficiency requirements for many specific problems. In this paper, an advanced hybrid surrogate model, namely, extended adaptive hybrid functions (E-AHF), is developed, which consists of two major components. The first part automatically filters out the poorly performing individual models and remains the appropriate ones based on the leave-one-out (LOO) cross-validation (CV) error. The second part calculates the adaptive weight factors for each individual surrogate model based on the baseline model and the estimated mean square error in a Gaussian process prediction. A large set of numerical experiments consisting of up to 40 test problems from one dimension to 16 dimensions are used to verify the accuracy and robustness of the proposed model. The results show that both the accuracy and the robustness of E-AHF have been remarkably improved compared with the individual surrogate models and multiple benchmark hybrid surrogate models. The computational time of E-AHF has also been considerately reduced compared with other hybrid models.

Copyright © 2018 by ASME
Your Session has timed out. Please sign back in to continue.


Queipo, N. V. , Haftka, R. T. , Shyy, W. , Goel, T. , Vaidyanathan R. , and Tucker, P. K. , 2005, “ Surrogate-Based Analysis and Optimization,” Prog. Aerosp. Sci., 41(1), pp. 1–28. [CrossRef]
Song, X. , Sun, G. , Li, G. , Gao, W. , and Li, Q. , 2013, “ Crashworthiness Optimization of Foam-Filled Tapered Thin-Walled Structure Using Multiple Surrogate Models,” Struct. Multidiscipli. Optim., 47(2), pp. 221–231.
Gorissen, D. , Couckuyt, I. , Demeester, P. , Dhaene, T. , and Crombecq, K. , 2010, “ A Surrogate Modeling and Adaptive Sampling Toolbox for Computer Based Design,” J. Mach. Learn. Res., 11(1), pp. 2051–2055.
Forrester, A. I. J. , Sóbester, A. , and Keane, A. J. , 2007, “ Multi-Fidelity Optimization Via Surrogate Modeling,” Proc. R. Soc. London A, 463(2088), pp. 3251–3269. [CrossRef]
Peri, D. , and Campana, E. F. , 2005, “ High-Fidelity Models and Multiobjective Global Optimization Algorithms in Simulation-Based Design,” J. Ship Res., 49(3), pp. 159–175.
Yang, J. , Zhan, Z. , Zheng, K. , Hu, J. , and Zheng, L. , 2016, “ Enhanced Similarity-Based Metamodel Updating Strategy for Reliability-Based Design Optimization,” Eng. Optim., 48(12), pp. 2026–2045. [CrossRef]
Ong, Y. S. , Nair, P. B. , Keane, A. J. , and Wong, K. W. , 2005, “ Surrogate-Assisted Evolutionary Optimization Frameworks for High-Fidelity Engineering Design Problems,” Knowledge Incorporation in Evolutionary Computation, Springer, Berlin, pp. 307–331. [CrossRef]
Draper, N. R. , and Smith, H. , 2014, Applied Regression Analysis, 3rd ed., Wiley, New York.
Matheron, G. , 1963, “ Principles of Geostatistics,” Econ. Geol., 58(8), pp. 1246–1266. [CrossRef]
Sacks, J. , Schiller, S. B. , and Welch, W. J. , 1989, “ Designs for Computer Experiments,” Technometrics, 31(1), pp. 41–47. [CrossRef]
Sacks, J. , Welch, W. J. , Mitchell, T. J. , and Wynn, H. P. , 1989, “ Design and Analysis of Computer Experiments,” Stat. Sci., 4(4), pp. 409–423. [CrossRef]
Fang, H. , and Horstemeyer, M. F. , 2006, “ Global Response Approximation With Radial Basis Functions,” Eng. Optim., 38(4), pp. 407–424. [CrossRef]
Vapnik, V. N. , and Vapnik, V. , 1998, Statistical Learning Theory, Wiley, New York.
Girosi, F. , 1998, “ An Equivalence Between Sparse Approximation and Support Vector Machines,” Neural Comput., 10(6), pp. 1455–1480. [CrossRef] [PubMed]
Zhou, X. J. , and Jiang, T. , 2016, “ Metamodel Selection Based on Stepwise Regression,” Struct. Multidiscip. Optim., 54(3), pp. 641–657. [CrossRef]
Goel, T. , Haftka, R. T. , Shyy, W. , and Queipo, N. V. , 2007, “ Ensemble of Surrogates,” Struct. Multidiscip. Optim., 33(3), pp. 199–216. [CrossRef]
Wang, G. G. , and Shan, S. , 2007, “ Review of Metamodeling Techniques in Support of Engineering Design Optimization,” ASME J. Mech. Des., 129(4), pp. 370–380. [CrossRef]
Simpson, T. W. , Mauery, T. M. , Korte, J. J. , and Mistree, F. , 2001, “ Kriging Models for Global Approximation in Simulation-Based Multidisciplinary Design Optimization,” AIAA J., 39(12), pp. 2233–2241. [CrossRef]
Clarke, S. M. , Griebsch, J. H. , and Simpson, T. W. , 2005, “ Analysis of Support Vector Regression for Approximation of Complex Engineering Analyses,” ASME J. Mech. Des., 127(6), pp. 1077–1087. [CrossRef]
Jin, R. , Chen, W. , and Simpson, T. W. , 2001, “ Comparative Studies of Metamodelling Techniques Under Multiple Modelling Criteria,” Struct. Multidiscip. Optim., 23(1), pp. 1–13. [CrossRef]
Fang, H. , Rais-Rohani, M. , Liu, Z. , and Horstemeyer, M. F. , 2005, “ A Comparative Study of Metamodeling Methods for Multiobjective Crashworthiness Optimization,” Comput. Struct., 83(25–26), pp. 2121–2136. [CrossRef]
Zerpa, L. E. , Queipo, N. V. , Pintos, S. , and Salager, J. L. , 2005, “ An Optimization Methodology of Alkaline–Surfactant–Polymer Flooding Processes Using Field Scale Numerical Simulation and Multiple Surrogates,” J. Pet. Sci. Eng., 47(3–4), pp. 197–208. [CrossRef]
Acar, E. , and Rais-Rohani, M. , 2009, “ Ensemble of Metamodels With Optimized Weight Factors,” Struct. Multidiscip. Optim., 37(3), pp. 279–294. [CrossRef]
Ferreira, W. G. , and Serpa, A. L. , 2016, “ Ensemble of Metamodels: The Augmented Least Squares Approach,” Struct. Multidiscip. Optim., 53(5), pp. 1019–1046. [CrossRef]
Viana, F. A. C. , Haftka, R. T. , and Steffen, V., Jr. , 2009, “ Multiple Surrogates: How Cross-Validation Errors Can Help Us to Obtain the Best Predictor,” Struct. Multidiscip. Optim., 39(4), pp. 439–457. [CrossRef]
Acar, E. , 2010, “ Various Approaches for Constructing an Ensemble of Metamodels Using Local Measures,” Struct. Multidiscip. Optim., 42(6), pp. 879–896. [CrossRef]
Zhang, J. , Chowdhury, S. , and Messac, A. , 2012, “ An Adaptive Hybrid Surrogate Model,” Struct. Multidiscip. Optim., 46(2), pp. 223–238. [CrossRef]
Liu, H. , Xu, S. , Wang, X. , Meng, J. , and Yang, S. , 2016, “ Optimal Weighted Pointwise Ensemble of Radial Basis Functions With Different Basis Functions,” AIAA J., 54(10), pp. 3117–3133. [CrossRef]
Forrester, A. , Sobester, A. , and Keane, A. , 2008, Engineering Design Via Surrogate Modelling: A Practical Guide, Wiley, Chichester, UK.
Lee, Y. , and Choi, D. , 2014, “ Pointwise Ensemble of Meta-Models Using υ Nearest Points Cross-Validation,” Struct. Multidiscip. Optim., 50(3), pp. 383–394. [CrossRef]
Glaz, B. , Goel, T. , Liu, L. , and Haftka, R. T. , 2007, “Application of a Weighted Average Surrogate Approach to Helicopter Rotor Blade Vibration Reduction,” AIAA Paper No. 2007-1898.
Gramacy, R. B. , and Lee, H. K. H. , 2012, “ Cases for the Nugget in Modeling Computer Experiments,” Stat. Comput., 22(3), pp. 713–722. [CrossRef]
Talgorn, B. , Kokkolaras, M. , and Digabel, S. L. , 2015, “ Statistical Surrogate Formulations for Simulation-Based Design Optimization,” ASME J. Mech. Des., 137(2), p. 021405. [CrossRef]
Mullur, A. A. , and Messac, A. , 2006, “ Metamodeling Using Extended Radial Basis Functions: A Comparative Approach,” Eng. Comput., 21(3), pp. 203–217. [CrossRef]
Cai, X. , Qiu, H. , Gao, L. , and Shao, X. , 2016, “ An Enhanced RBF-HDMR Integrated With an Adaptive Sampling Method for Approximating High Dimensional Problems in Engineering Design,” Struct. Multidiscip. Optim., 53(6), pp. 1209–1229. [CrossRef]
McKay, M. D. , Beckman, R. J. , and Conover, W. J. , 2000, “ A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code,” Technometrics, 42(1), pp. 55–61. [CrossRef]
Viana, F. A. C. , 2010, “SURROGATES Toolbox User's Guide,” Gainesville, FL.
Brabanter, K. D. , Karsmakers, P. , Ojeda, F. , Alzate, C. , Brabanter, J. D. , Pelckmans, K. , Moor, B. D. , Vandewalle, J. , and Suykens, J. A. K. , 2011, “LS-SVMlab Toolbox User's Guide. Version 1.8,” ESAT-SISTA, Katholieke Universiteit Leuven, Belgium, Technical Report No. 10-146.
Suykens, J. A. K. , and Vandewalle, J. , 1999, “ Least Squares Support Vector Machine Classifiers,” Neural Process. Lett., 9(3), pp. 293–300. [CrossRef]


Grahic Jump Location
Fig. 1

The framework of the AHF surrogate model [27]

Grahic Jump Location
Fig. 2

Different surrogate models based on five training points

Grahic Jump Location
Fig. 3

Error region and values from the Gaussian process

Grahic Jump Location
Fig. 4

Adaptive weight factors for different individual models

Grahic Jump Location
Fig. 5

Effect of DoE sets on the accuracy of E-AHF and Kriging models

Grahic Jump Location
Fig. 6

R-square comparison of different surrogate models for 40 test problems: (a) compared with individual models and (b) compared with hybrid models

Grahic Jump Location
Fig. 7

R-square comparison under different sample sets: (a) compared with individual models and (b) compared with hybrid models

Grahic Jump Location
Fig. 8

R-square comparison for different-dimensional test problems: (a) compared with individual models and (b) compared with hybrid models

Grahic Jump Location
Fig. 9

Comparison of computational time of hybrid models

Grahic Jump Location
Fig. 10

Effect of threshold value on the E-AHF model




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In