Research Papers

Methodology for Global Optimization of Computationally Expensive Design Problems

[+] Author and Article Information
Stefanos Koullias

Aerospace Systems Design Laboratory,
School of Aerospace Engineering,
Georgia Institute of Technology,
Atlanta, GA 30332
e-mail: skoullias3@gmail.com

Dimitri N. Mavris

Aerospace Systems Design Laboratory,
School of Aerospace Engineering,
Georgia Institute of Technology,
Atlanta, GA 30332
e-mail: dimitri.mavris@aerospace.gatech.edu

In the fictitious setup of describing a deterministic function y(·) as a realization of a Gaussian random process, the deterministic observations y as well as the value y(x) are random variables, but in this manuscript they are written in the lower case to remind the reader that they are not true random variables.

It is understood that the expectation EI(x) is the conditional expectation EI|y(x).

Equality constraints can be written as two inequality constraints.

Henceforth, all parameters of the constraint function GP model cj shall be indicated using the relevant variables thus far but with a j subscript, where j=1,,m

The implementation is identical to EGO except for the ISC. The P-algorithm maximizes the probability of improvement.

1Corresponding author.

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received September 2, 2013; final manuscript received April 3, 2014; published online June 2, 2014. Assoc. Editor: Michael Kokkolaras.

J. Mech. Des 136(8), 081007 (Jun 02, 2014) (12 pages) Paper No: MD-13-1387; doi: 10.1115/1.4027493 History: Received September 02, 2013; Revised April 03, 2014

The design of unconventional systems requires early use of high-fidelity physics-based tools to search the design space for improved and potentially optimum designs. Current methods for incorporating these computationally expensive tools into early design for the purpose of reducing uncertainty are inadequate due to the limited computational resources that are available in early design. Furthermore, the lack of finite difference derivatives, unknown design space properties, and the possibility of code failures motivates the need for a robust and efficient global optimization (EGO) algorithm. A novel surrogate model-based global optimization algorithm capable of efficiently searching challenging design spaces for improved designs is presented. The algorithm, called fBcEGO for fully Bayesian constrained EGO, constructs a fully Bayesian Gaussian process (GP) model through a set of observations and then uses the model to make new observations in promising areas where improvements are likely to occur. This model remedies the inadequacies of likelihood-based approaches, which may provide an incomplete inference of the underlying function when function evaluations are expensive and therefore scarce. A challenge in the construction of the fully Bayesian GP model is the selection of the prior distribution placed on the model hyperparameters. Previous work employs static priors, which may not capture a sufficient number of interpretations of the data to make any useful inferences about the underlying function. An iterative method that dynamically assigns hyperparameter priors by exploiting the mechanics of Bayesian penalization is presented. fBcEGO is incorporated into a methodology that generates relatively few infeasible designs and provides large reductions in the objective function values of design problems. This new algorithm, upon implementation, was found to solve more nonlinearly constrained algebraic test problems to higher accuracies relative to the global minimum than other popular surrogate model-based global optimization algorithms and obtained the largest reduction in the takeoff gross weight objective function for the case study of a notional 70-passenger regional jet when compared with competing design methods.

Copyright © 2014 by ASME
Your Session has timed out. Please sign back in to continue.


Raymer, D. P., 2006, Aircraft Design: A Conceptual Approach, 4th ed., AIAA Education Series, American Institute of Aeronautics and Astronautics, Reston, VA.
Rawson, K., and Tupper, E., 2001, Basic Ship Theory: Volume 1, 5th ed., Butterworth-Heinemann, Boston, MA.
Drela, M., and Youngren, H., 2001, XFOIL 6.94, Cambridge, MA, http://web.mit.edu/drela/Public/web/xfoil/
Rogers, S. E., Roth, K., Cao, H. V., Slotnick, J. P., Whitlock, M., Nash, S. M., and Baker, M. D., 2001, “Computation of Viscous Flow for a Boeing 777 Aircraft in Landing Configuration,” J. Aircraft, 38(6), pp. 1060–1068. [CrossRef]
Rizzi, A., 2011, “Modeling and Simulation Aircraft Stability and Control—The SimSAC Project,” Prog. Aerosp. Sci., 47(8), pp. 573–588. [CrossRef]
Jones, D. R., Schonlau, M., and Welch, W. J., 1998, “Efficient Global Optimization of Expensive Black-Box Functions,” J. Global Optim., 13(4), pp. 455–492. [CrossRef]
Forrester, A. I. J., 2004, “Efficient Global Aerodynamic Optimisation Using Expensive Computational Fluid Dynamics Simulations,” Ph.D. thesis, University of Southampton, Southampton, UK.
Alexandrov, N., Lewis, R., Gumbert, C., Green, L., and Newman, P., 2000, “Optimization With Variable-Fidelity Models Applied to Wing Design,” 38th Aerospace Sciences Meeting & Exhibit, Reno, NV, Paper No. AIAA-2000-0841.
Audet, C., and Dennis, J. E., Jr., 2004, “A Pattern Search Filter Method for Nonlinear Programming Without Derivatives,” SIAM J. Optim., 14(4), pp. 980–1010. [CrossRef]
Mason, W., Knill, D., Giunta, A., Grossman, B., and Watson, L., 1998, “Getting the Full Benefits of CFD in Conceptual Design,” 16th AIAA Applied Aerodynamics Conference, Albuquerque, NM, Paper No. AIAA 98-2513.
Myers, R. H., and Montgomery, D. C., 2002, Response Surface Methodology: Process and Product Optimization Using Designed Experiments, John Wiley & Sons, Inc., New York.
Jones, D. R., 2001, “A Taxonomy of Global Optimization Methods Based on Response Surfaces,” J. Global Optim., 21(4), pp. 345–383. [CrossRef]
Keane, A. J., and Nair, P. B., 2005, Computational Approaches for Aerospace Design: The Pursuit of Excellence, John Wiley & Sons, Ltd., Hoboken, NJ.
Forrester, A. I., Sóbester, A., and Keane, A. J., 2008, Engineering Design via Surrogate Modelling—A Practical Guide, John Wiley & Sons, Inc., Hoboken, NJ.
Osborne, M. A., 2010, “Bayesian Gaussian Processes for Sequential Prediction, Optimisation, and Quadrature,” Ph.D. thesis, University of Oxford, Oxford, UK.
Benassi, R., Bect, J., and Vazquez, E., 2011, “Robust Gaussian Process-Based Global Optimization Using a Fully Bayesian Expected Improvement Criterion,” 5th International Conference on Learning and Intelligent Optimization (LION 5), C. A. C. Coello, ed., Vol. 6683 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, pp. 176–190.
Conn, A. R., Scheinberg, K., and Vicente, L. N., 2009, Introduction to Derivative-Free Optimization, SIAM, Philadelphia, PA.
Conn, A. R., Gould, N. I., and Toint, P. L., 2000, Trust-Region Methods, SIAM, Philadelphia, PA.
Vaz, A. F., and Vicente, L. N., 2007, “A Particle Swarm Pattern Search Method for Bound Constrained Global Optimization,” J. Global Optim., 39(2), pp. 197–219. [CrossRef]
Digabel, S. L., 2011, “Algorithm 909: NOMAD: Nonlinear Optimization With the MADS Algorithm,” ACM Trans. Math. Software, 37(4), pp. 44:1–44:15. [CrossRef]
Schonlau, M., Welch, W., and Jones, D. R., 1998, “Global Versus Local Search in Constrained Optimization of Computer Models,” New Developments and Applications in Experimental Design, Vol. 34, N.Flournoy, W. F.Rosenberger, and W. K.Wong, eds., Institute of Mathematical Statistics, Hayward, CA, pp. 11–25.
Rasmussen, C. E., and Williams, C. K. I., 2006, Gaussian Processes for Machine Learning, The MIT Press, Cambridge, MA.
Santner, T. J., Williams, B. J., and Notz, W. I., 2003, The Design and Analysis of Computer Experiments, Springer, New York.
Rasmussen, C., and Ghahramani, Z., 2003, “Bayesian Monte Carlo,” Advances in Neural Information Processing Systems, Vol. 15, S. Becker, S. Thrun, and K. Obermayer, eds., MIT Press, Cambridge, MA, pp. 505–512.
Osborne, M. A., Garnett, R., and Roberts, S. J., 2009, “Gaussian Processes for Global Optimization,” 3rd International Conference on Learning and Intelligent Optimization (LION3), Trento, Italy.
Handcock, M. S., and Stein, M. L., 1993, “A Bayesian Analysis of Kriging,” Technometrics, 35(4), pp. 403–410. [CrossRef]
Koullias, S., 2013, “Methodology for Global Optimization of Computationally Expensive Design Problems,” Ph.D. thesis, Georgia Institute of Technology, Atlanta, GA.
MacKay, D. J., 2003, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, Cambridge, UK.
Jones, D., Perttunen, C., and Stuckman, B., 1993, “Lipschitzian Optimization Without the Lipschitz Constant,” J. Optim. Theory Appl., 79(1), pp. 157–181. [CrossRef]
Quttineh, N.-H., and Holmström, K., 2009, “The Influence of Experimental Designs on the Performance of Surrogate Model Based Costly Global Optimization Solvers,” Stud. Inf. Control, 18(1), pp. 87–95.
O'Hara, J. J., Stump, G. M., Yukish, M. A., Harris, E. N., Hanowski, G. J., and Carty, A., 2007, “Advanced Visualization Techniques for Trade Space Exploration,” 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, April 23–26, pp. 1–6, Paper No. AIAA 2007-1878.
Holden, C. M., and Keane, A. J., 2004, “Visualization Methodologies in Aircraft Design,” 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, 30 Aug.–1 Sept., pp. 1–13, Paper No. AIAA 2004-4449.
Khompatraporn, C., Zabinsky, Z. B., and Pintér, J. D., 2005, “Comparative Assessment of Algorithms and Software for Global Optimization,” J. Global Optim., 31(4), pp. 613–633. [CrossRef]
Floudas, C., and Pardalos, P., 1990, A Collection of Test Problems for Constrained Global Optimization Algorithms (Lecture Notes in Computer Science) Vol. 455, Springer-Verlag, Berlin, Germany.
Floudas, C. A., Pardalos, P. M., Adjiman, C. S., Esposito, W. R., Gümüs, Z. H., Harding, S. T., Klepeis, J. L., Meyer, C. A., and Schweiger, C. A., 1999, Handbook of Test Problems in Local and Global Optimization, Kluwer Academic, Dordrecht, The Netherlands.
Hock, W., and Schittkowski, K., 1981, Test Examples for Nonlinear Programming Codes, Springer-Verlag, Berlin.
Sasena, M., Papalambros, P., and Goovaerts, P., 2002, “Global Optimization of Problems With Disconnected Feasible Regions via Surrogate Modeling,” 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Atlanta, GA, Paper No. AIAA-2002-5573.
Moré, J. J., and Wild, S. M., 2009, “Benchmarking Derivative-Free Optimization Algorithms,” SIAM J. Optim., 20(1), pp. 172–191. [CrossRef]
Quttineh, N.-H., and Holmström, K., 2009, Implementation of a One-Stage Efficient Global Optimization (EGO) Algorithm, Mälardalen University, School of Education, Culture and Communication, Technical Report, Research Reports MDH/UKK 2009-2, http://www.mai.liu.se/~niqut/publications.html
Gutmann, H., 2001, “A Radial Basis Function Method for Global Optimization,” J. Global Optim., 19(3), pp. 201–227. [CrossRef]
Regis, R. G., and Shoemaker, C. A., 2005, “Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions,” J. Global Optim., 31(1), pp. 153–171. [CrossRef]
Nocedal, J., and Wright, S. J., 2006, Numerical Optimization, 2nd ed., Springer, New York.
Vanderplaats, G. N., 2005, Numerical Optimization Techniques for Engineering Design, 4th ed., Garret N. Vanderplaats, Colorado Springs, CO.
Jones, D. R., 2009, “Direct Global Optimization Algorithm,” Encyclopedia of Optimization, 2nd ed., C. A.Floudas and P. M.Pardalos, eds., Springer, New York, pp. 725–735.
Abramowitz, M., and Stegun, I. A., eds., 1972, Handbook of Mathematical Functions, 10th ed., Vol. 55, (Applied Mathematics Series), National Bureau of Standards, New York.
Cox, S. E., Haftka, R. T., Barker, C. A., Grossman, B., Mason, W. H., and Watson, L. T., 2001, “A Comparison of Global Optimization Methods for the Design of a High-Speed Civil Transport,” J. Global Optim., 21, pp. 415–433. [CrossRef]
Lee, K.-Y., and Roh, M.-I., 2001, “An Efficient Genetic Algorithm Using Gradient Information for Ship Structural Design Optimization,” Ship Tech. Res./Schiffstechnik, 48, pp. 161–170.
McCullers, L. A., 2004, FLOPS: Flight Optimization System Release 6.12, NASA Langley Research Center, Hampton, VA.
Feagin, R. C., and Morrison, W. D., 1978, “Delta Method, an Empirical Drag Buildup Technique, NASA, Technical Report No. CR-151971.
Sommer, S. C., and Short, B. J., 1955, “Free-Flight Measurements of Turbulent-Boundary-Layer Skin Friction in the Presence of Severe Aerodynamic Heating at High Mach Numbers From 2.8 to 7.0, NASA, Technical Report No. TN-3391.
Federal Aviation Administration, 2013, “FAA Regulations,” Retrieved on May 26, 2013, http://www.faa.gov/regulations_policies/faa_regulations


Grahic Jump Location
Fig. 1

MLE fit (top) compared with fully Bayesian fit (bottom). Shaded area represents an uncertainty region of ±2 s.

Grahic Jump Location
Fig. 2

Bayesian penalization of unnecessarily complex models and overly simple models. Conditional models (solid lines) and weighted sum model (dotted line) through four observations (+).

Grahic Jump Location
Fig. 3

Results of nonlinearly constrained tests after N simplex gradients. N = 5, 10, and 20 shown in top, middle, and bottom panels, respectively.

Grahic Jump Location
Fig. 4

Results of nonlinearly constrained tests for all N in terms of equality constraint violation

Grahic Jump Location
Fig. 5

Mission profile for notional 70-passenger jet

Grahic Jump Location
Fig. 6

Baseline (solid) versus optimized (wireframe) configuration




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In