Research Papers

A Robust Error-Pursuing Sequential Sampling Approach for Global Metamodeling Based on Voronoi Diagram and Cross Validation

[+] Author and Article Information
Shengli Xu

School of Energy and Power Engineering,
Dalian University of Technology,
Dalian 116024, China
e-mail: xusl@dlut.edu.cn

Haitao Liu

School of Energy and Power Engineering,
Dalian University of Technology,
Dalian 116024, China
e-mail: lht@mail.dlut.edu.cn

Xiaofang Wang

School of Energy and Power Engineering,
Dalian University of Technology,
Dalian 116024, China
e-mail: dlwxf@dlut.edu.cn

Xiaomo Jiang

School of Energy and Power Engineering,
Dalian University of Technology,
Dalian 116024, China
e-mail: jiang.xiaomo@gmail.com

1Corresponding author.

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received August 2, 2013; final manuscript received February 23, 2014; published online April 28, 2014. Assoc. Editor: Xiaoping Du.

J. Mech. Des 136(7), 071009 (Apr 28, 2014) (10 pages) Paper No: MD-13-1334; doi: 10.1115/1.4027161 History: Received August 02, 2013; Revised February 23, 2014

Surrogate models are widely used in simulation-based engineering design and optimization to save the computing cost. The choice of sampling approach has a great impact on the metamodel accuracy. This article presents a robust error-pursuing sequential sampling approach called cross-validation (CV)-Voronoi for global metamodeling. During the sampling process, CV-Voronoi uses Voronoi diagram to partition the design space into a set of Voronoi cells according to existing points. The error behavior of each cell is estimated by leave-one-out (LOO) cross-validation approach. Large prediction error indicates that the constructed metamodel in this Voronoi cell has not been fitted well and, thus, new points should be sampled in this cell. In order to rapidly improve the metamodel accuracy, the proposed approach samples a Voronoi cell with the largest error value, which is marked as a sensitive region. The sampling approach exploits locally by the identification of sensitive region and explores globally with the shift of sensitive region. Comparative results with several sequential sampling approaches have demonstrated that the proposed approach is simple, robust, and achieves the desired metamodel accuracy with fewer samples, that is needed in simulation-based engineering design problems.

Copyright © 2014 by ASME
Topics: Design , Errors , Simulation
Your Session has timed out. Please sign back in to continue.


Cressie, N., 1988, “Spatial Prediction and Ordinary Kriging,” Math. Geol., 20(4), pp. 405–421. [CrossRef]
Dyn, N., Levin, D., and Rippa, S., 1986, “Numerical Procedures for Surface Fitting of Scattered Data by Radial Functions,” SIAM J. Sci. Stat. Comput., 7(2), pp. 639–659. [CrossRef]
Fang, H., and Horstemeyer, M. F., 2006, “Global Response Approximation With Radial Basis Functions,” Eng. Optim., 38(4), pp. 407–424. [CrossRef]
Friedman, J. H., 1991, “Multivariate Adaptive Regression Splines,” Ann. Stat., 19(1), pp. 1–67. [CrossRef]
Wang, G. G., and Shan, S., 2007, “Review of Metamodeling Techniques in Support of Engineering Design Optimization,” ASME J. Mech. Des., 129(4), pp. 370–380. [CrossRef]
McKay, M. D., Beckman, R. J., and Conover, W. J., 1979, “Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code,” Technometrics, 21(2), pp. 239–245.
Owen, A. B., 1992, “Orthogonal Arrays for Computer Experiments, Integration and Visualization,” Stat. Sin., 2(2), pp. 439–452.
Ye, K. Q., Li, W., and Sudjianto, A., 2000, “Algorithmic Construction of Optimal Symmetric Latin Hypercube Designs,” J. Stat. Plann. Inference, 90(1), pp. 145–159. [CrossRef]
Grosso, A., Jamali, A., and Locatelli, M., 2009, “Finding Maximin Latin Hypercube Designs by Iterated Local Search heuristics,” Eur. J. Oper. Res., 197(2), pp. 541–547. [CrossRef]
Viana, F. A., Venter, G., and Balabanov, V., 2010, “An Algorithm for Fast Optimal Latin Hypercube Design of Experiments,” Int. J. Numer. Methods Eng., 82(2), pp. 135–156.
Zhu, H., Liu, L., Long, T., and Peng, L., 2012, “A Novel Algorithm of Maximin Latin Hypercube Design Using Successive Local Enumeration,” Eng. Optim., 44(5), pp. 551–564. [CrossRef]
Jin, R., Chen, W., and Sudjianto, A., 2002, “On Sequential Sampling for Global Metamodeling in Engineering Design,” Proceedings of ASME Design Automation Conference, Montreal, Canada, September 29–October 2, 2002, ASME, pp. 539–548. [CrossRef]
Jones, D. R., Schonlau, M., and Welch, W. J., 1998, “Efficient Global Optimization of Expensive Black-Box Functions,” J. Global Optim., 13(4), pp. 455–492. [CrossRef]
Booker, A. J., Dennis, J., Jr., Frank, P. D., Serafini, D. B., Torczon, V., and Trosset, M. W., 1999, “A Rigorous Framework for Optimization of Expensive Functions by Surrogates,” Struct. Optim., 17(1), pp. 1–13. [CrossRef]
Cox, D. D., and John, S., 1997, “SDO: A Statistical Method for Global Optimization,” In N. Alexandrov and M. Y. Hussaini, Multidisciplinary Design Optimization: State of the Art, SIAM, Philadelphia, pp. 315–329.
Trosset, M. W., and Torczon, V., 1997, “Numerical Optimization Using Computer Experiments,” NASA Langley Research Center, Technical Report No. 97-38.
Johnson, M. E., Moore, L. M., and Ylvisaker, D., 1990, “Minimax and Maximin Distance Designs,” J. Stat. Plann. Inference, 26(2), pp. 131–148. [CrossRef]
Audze, P., and Eglais, V., 1977, “New Approach for Planning Out of Experiments,” Prob. Dyn. Strengths, 35, pp. 104–107.
Fang, K. T., Ma, C. X., and Winker, P., 2002, “Centered L2-Discrepancy of Random Sampling and Latin Hypercube Design, and Construction of Uniform Designs,” Math. Comput., 71(237), pp. 275–296. [CrossRef]
Iman, R. L., and Conover, W., 1980, “Small Sample Sensitivity Analysis Techniques for Computer Models. With an Application to Risk Assessment,” Commun. Stat. Theory Methods, 9(17), pp. 1749–1842. [CrossRef]
Xiong, F., Xiong, Y., Chen, W., and Yang, S., 2009, “Optimizing Latin Hypercube Design for Sequential Sampling of Computer Experiments,” Eng. Optim., 41(8), pp. 793–810. [CrossRef]
Crombecq, K., Laermans, E., and Dhaene, T., 2011, “Efficient Space-Filling and Non-Collapsing Sequential Design Strategies for Simulation-Based Modeling,” Eur. J. Oper. Res., 214(3), pp. 683–696. [CrossRef]
Watson, A. G., and Barnes, R. J., 1995, “Infill Sampling Criteria to Locate Extremes,” Math. Geol., 27(5), pp. 589–608. [CrossRef]
Farhang Mehr, A., and Azarm, S., 2005, “Bayesian Meta‐Modelling of Engineering Design Simulations: A Sequential Approach With Adaptation to Irregularities in the Response Behaviour,” Int. J. Numer. Methods Eng., 62(15), pp. 2104–2126. [CrossRef]
Li, G., Aute, V., and Azarm, S., 2010, “An Accumulative Error Based Adaptive Design of Experiments for Offline Metamodeling,” Struct. Multidiscip. Optim., 40(1–6), pp. 137–155. [CrossRef]
Forrester, A., Sóbester, A., and Keane, A., 2008, Engineering Design via Surrogate Modelling: A Practical Guide, John Wiley & Sons, Chichester, UK.
Lin, Y., 2004, “An Efficient Robust Concept Exploration Method and Sequential Exploratory Experimental Design,” Ph.D. thesis, Georgia Institute of Technology.
Turner, C. J., Crawford, R. H., and Campbell, M. I., 2007, “Multidimensional Sequential Sampling for NURBs-Based Metamodel Development,” Eng. Comput., 23(3), pp. 155–174. [CrossRef]
Aute, V., Saleh, K., Abdelaziz, O., Azarm, S., and Radermacher, R., 2013, “Cross-Validation Based Single Response Adaptive Design of Experiments for Kriging Metamodeling of Deterministic Computer Simulations,” Struct. Multidiscip. Optim., 48(3), pp. 581–605. [CrossRef]
Crombecq, K., Gorissen, D., Deschrijver, D., and Dhaene, T., 2011, “A Novel Hybrid Sequential Design Strategy for Global Surrogate Modeling of Computer Experiments,” SIAM J. Sci. Comput., 33(4), pp. 1948–1974. [CrossRef]
Aurenhammer, F., 1991, “Voronoi diagrams—a survey of a fundamental geometric data structure,” J. ACM Comput. Surveys, 23(3), pp. 345–405. [CrossRef]
Couckuyt, I., Forrester, A., Gorissen, D., De Turck, F., and Dhaene, T., 2012, “Blind Kriging: Implementation and Performance Analysis,” Adv. Eng. Software, 49, pp. 1–13. [CrossRef]


Grahic Jump Location
Fig. 1

A set of 2D samples and the corresponding Voronoi cells

Grahic Jump Location
Fig. 2

A Voronoi cell described by a set of random points, where the circle represents the existing sample and the triangles represent the random points

Grahic Jump Location
Fig. 3

The prediction errors of some 2D samples and the corresponding Voronoi cells, where bigger circle implies larger error

Grahic Jump Location
Fig. 4

A 1D function and a metamodel through five initial samples

Grahic Jump Location
Fig. 5

Samples generated by CV-Voronoi, ACE, and SFCVT for the 1D test case

Grahic Jump Location
Fig. 6

The S values of CV-Voronoi, ACE, and SFCVT during the sampling process

Grahic Jump Location
Fig. 7

Illustration of local exploitation and global exploration of the proposed approach

Grahic Jump Location
Fig. 8

The trail of the additional 15 samples for the aforementioned 1D case by the proposed approach

Grahic Jump Location
Fig. 9

Convergence histories (mean + SD) of CV-Voronoi, SFCVT, and ACE for a 1D test case

Grahic Jump Location
Fig. 10

Two runs for function Peaks with (a) CV-Voronoi (37 samples) and (b) LOLA-Voronoi (47 samples), separately

Grahic Jump Location
Fig. 11

Two runs for function Easom with (a) CV-Voronoi (83 samples) and (b) LOLA-Voronoi (126 samples), separately

Grahic Jump Location
Fig. 12

Convergence histories (mean + SD) of different approaches for functions: (a) Peaks, (b) Shekel, and (c) Ackley10.

Grahic Jump Location
Fig. 13

Required number of samples (mean + SD) to reach specified RMSE by CV-Voronoi with different numbers of initial samples for functions (a) Peaks, (b) Shekel, and (c) Hart6.



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In