Research Papers

High Dimensional Model Representation With Principal Component Analysis

[+] Author and Article Information
Kambiz Haji Hajikolaei

e-mail: khajihaj@sfu.ca

G. Gary Wang

e-mail: gary_wang@sfu.ca
Product Design and Optimization
Laboratory (PDOL),
School of Mechatronic Systems Engineering,
Simon Fraser University,
250-13450 102 Avenue,
Surrey, BC V3T0A3, Canada

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received March 13, 2013; final manuscript received September 6, 2013; published online October 17, 2013. Assoc. Editor: Michael Kokkolaras.

J. Mech. Des 136(1), 011003 (Oct 17, 2013) (11 pages) Paper No: MD-13-1120; doi: 10.1115/1.4025491 History: Received March 13, 2013; Revised September 06, 2013

In engineering design, spending excessive amount of time on physical experiments or expensive simulations makes the design costly and lengthy. This issue exacerbates when the design problem has a large number of inputs, or of high dimension. High dimensional model representation (HDMR) is one powerful method in approximating high dimensional, expensive, black-box (HEB) problems. One existing HDMR implementation, random sampling HDMR (RS-HDMR), can build an HDMR model from random sample points with a linear combination of basis functions. The most critical issue in RS-HDMR is that calculating the coefficients for the basis functions includes integrals that are approximated by Monte Carlo summations, which are error prone with limited samples and especially with nonuniform sampling. In this paper, a new approach based on principal component analysis (PCA), called PCA-HDMR, is proposed for finding the coefficients that provide the best linear combination of the bases with minimum error and without using any integral. Several benchmark problems of different dimensionalities and one engineering problem are modeled using the method and the results are compared with RS-HDMR results. In all problems with both uniform and nonuniform sampling, PCA-HDMR built more accurate models than RS-HDMR for a given set of sample points.

Copyright © 2014 by ASME
Your Session has timed out. Please sign back in to continue.


Wang, G. G., and Shan, S., 2007, “Review of Metamodeling Techniques in Support of Engineering Design Optimization,” ASME J. Mech. Des., 129(4), pp. 370–380. [CrossRef]
Cresssie, N., 1988, “Spatial Prediction and Ordinary Kriging,” Math. Geol., 20(4), pp. 405–421. [CrossRef]
Fang, H., and Horstemeyer, M. F., 2006, “Global Response Approximation With Radial Basis Functions,” J. Eng. Optim., 38(4), pp. 407–424. [CrossRef]
Papadrakakis, M., Lagaros, M., and Tsompanakis, Y., 1998, “Structural Optimization Using Evolution Strategies and Neural Networks,” Comput. Methods Appl. Mech. Eng., 156(1–4), pp. 309–333. [CrossRef]
Friedman, J. H., 1991, “Multivariate Adaptive Regressive Splines,” Ann. Stat., 19(1), pp. 1–67. [CrossRef]
Shan, S., and Wang, G. G., 2010, “Survey of Modeling and Optimization Strategies to Solve High Dimensional Design Problems With Computationally Expensive Black-Box Functions,” Struct. Multidiscip. Optim., 41(2), pp. 219–241. [CrossRef]
Sobol, I. M., 1993, “Sensitivity Estimates for Nonlinear Mathematical Models,” Math. Modell. Comput. Exp., 1(4), pp. 407–414.
Rabitz, H., and Alis, O. F., 1999, “General Foundation of High Dimensional Model Representation,” J. Math. Chem., 25, pp. 197–233. [CrossRef]
Li, G., Rosenthal, C., and Rabitz, H., 2001, “High Dimensional Model Representations,” J. Phys. Chem., 105(33), pp. 7765–7777. [CrossRef]
Wang, H., Tang, L., and Li, G. Y., 2011, “Adaptive MLSHDMR Metamodeling Techniques for High Dimensional Problems,” Exp. Syst. Appl., 38, pp. 14117–14126.
Shan, S., and Wang, G. G., 2010, “Metamodeling for High Dimensional Simulation-Based Design Problems,” ASME J. Mech. Des., 132(5), pp. 1–11. [CrossRef]
Shan, S., and Wang, G. G., 2011, “Turning Black Box Into White Function,” ASME J. Mech. Des., 133(3), p. 031003. [CrossRef]
Li, G., Schoendorf, J., Ho, T., and Rabitz, H., 2004, “Multicut-HDMR With an Application to an Ionospheric Model,” J. Comput. Chem., 25, pp. 1149–1156. [CrossRef] [PubMed]
Tunga, M. A., and Demiralp, M., 2006, “Hybrid High Dimensional Model Representation (HHDMR) on the Partitioned Data,” J. Comput. Appl. Math., 185, pp. 107–132. [CrossRef]
Tunga, M. A., and Demiralp, M., 2008, “Introductory Steps for an Indexing Based HDMR Algorithm: Lumping HDMR,” 1st WSEAS International Conference on Multivariate Analysis and its Application in Science and Engineering, MAASE’08, May 27–30, Istanbul, Turkey, pp. 129–135.
Tunga, M. A., 2011, “An Approximation Method to Model Multivariate Interpolation Problems: Indexing HDMR,” Math. Comput. Model., 53, pp. 1970–1982. [CrossRef]
Li, G., and Rabitz, H., 2007, “Regularized Random-Sampling High Dimensional Model Representation (RS-HDMR),” J. Math. Chem., 43(3), pp. 1207–1232. [CrossRef]
Thomas, P. S., Somers, M. F., Hoekstra, A. W., and Kroes, G. J., 2012, “Chebyshev High-Dimensional Model Representation (Chebyshev-HDMR) Potentials: Application to Reactive Scattering of H2 from Pt(111) and Cu(111) Surfaces,” Phys. Chem. Chem. Phys., 14, pp. 8628–8643. [CrossRef]
Kaya, H., Kaplan, M., and Saygin, H., 2004, “A Recursive Algorithm for Finding HDMR Terms for Sensitivity Analysis,” Comput. Phys. Commun., 158, pp. 106–112. [CrossRef]
Alis, O. F., and Rabitz, H., 2001, “Efficient Implementation of High Dimensional Model Representations,” J. Math. Chem., 29(2), pp. 127–142. [CrossRef]
Hajikolaei, K. H., and Wang, G. G., 2012, “Adaptive Orthonormal Basis Functions for High Dimensional Metamodeling With Existing Sample Points,” Proceedings of the ASME 2012 International Design Engineering Technical Conference and Computers and Information in Engineering Conference, DETC2012-70480, Aug. 12–15, Chicago, IL.
Abdi, H., and Williams, L. J., 2010, “Principal Component Analysis,” Wiley Interdiscip. Rev. Comput. Stat., 2(4), pp. 433–459. [CrossRef]
Li, G., Wang, S. W., Rabitz, H., Wang, S., and Jaffé, P., 2002, “Global Uncertainty Assessments by High Dimensional Model Representations (HDMR),” Chem. Eng. Sci., 57(21), pp. 4445–4460. [CrossRef]
Li, G., Wang, S. W., and Rabitz, H., 2002, “Practical Approaches to Construct RS-HDMR Component Functions,” J. Phys. Chem., 106, pp. 8721–8733. [CrossRef]
Li, G., Artamonov, M., Rabitz, H., Wang, S., Georgopoulos, P. G., and Demirlap, M., 2002, “High-Dimensional Model Representations Generated from Low Order Terms—lp-RS-HDMR,” J. Comput. Chem., 24(5), pp. 647–656. [CrossRef]
Li, G., Hu, J., Wang, Sh., Georgopoulos, P. G., Schoendorf, J., and Rabitz, H., 2006, “Random Sampling-High Dimensional Model Representation (RS-HDMR) and Orthogonality of its Different Order Component Functions,” J. Phys. Chem., 110, pp. 2474–2485. [CrossRef]
Li, G., Rabitz, H., Wang, S., and Georgopoulos, P. G., 2002, “Correlation Method for Variance Reduction of Monte Carlo Integration in RS-HDMR,” J. Comput. Chem., 24(3), pp. 277–283. [CrossRef]
Li, G., and Rabitz, H., 2006, “Ratio Control Variate Method for Efficiently Determining High-Dimensional Model Representations,” J. Comput. Chem., 27, pp. 1112–1118. [CrossRef] [PubMed]
Pearson, K., 1901, “On Lines and Planes of Closest Fit to Systems of Points in Space,” Philos. Mag., 6, pp. 559–572.
Hotelling, H., 1933, “Analysis of a Complex of Statistical Variables Into Principal Components,” J. Educ. Psychol., 25, pp. 417–441. [CrossRef]
Hock, W., and Schittkowski, K., 1980, “Test Examples for Nonlinear Programming Codes,” J. Optim. Theory Appl., 30(1), pp. 127–129. [CrossRef]
Schittkowski, K., 1987, More Test Examples for Nonlinear Programming Codes, Springer-Verlag, New York.
Whitney, D. E., 2004, Mechanical Assemblies, Oxford University Press, New York.
3DCS Variation Analyst, 2013, Available at: www.3DCS.com


Grahic Jump Location
Fig. 1

The geometric representation of PCA

Grahic Jump Location
Fig. 2

R-square values of the first ten benchmark functions (average of 20 runs, sampling type 1)

Grahic Jump Location
Fig. 3

RAAE values of the first ten benchmark functions (average of 20 runs, sampling type 1)

Grahic Jump Location
Fig. 4

RMAE values of the first ten benchmark functions (average of 20 runs, sampling type 1)

Grahic Jump Location
Fig. 5

RAAE values of the first ten benchmark functions (average of 20 runs, sampling type 2)

Grahic Jump Location
Fig. 6

RMAE values of the first ten benchmark functions (average of 20 runs, sampling type 2)

Grahic Jump Location
Fig. 7

Sine function (solid), normal PCA-HDMR approximation (dash-dotted), PCA-HDMR with weight 10 after x≥0 (dashed), and PCA-HDMR with weight 1000 after x≥0 (dotted)

Grahic Jump Location
Fig. 8

PCA-HDMR results using different number of components

Grahic Jump Location
Fig. 9

Three-part assembly problem and the related fixtures [33]




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In