Skip to main content

Engineering Design via Surrogate Modelling: A Practical Guide




Engineering Design via Surrogate Modelling: A Practical Guide

Alexander Forrester, András Sobester, Andy Keane

ISBN: 978-0-470-06068-1 July 2008 228 Pages



Surrogate models expedite the search for promising designs by standing in for expensive design evaluations or simulations. They provide a global model of some metric of a design (such as weight, aerodynamic drag, cost, etc.), which can then be optimized efficiently.

Engineering Design via Surrogate Modelling is a self-contained guide to surrogate models and their use in engineering design. The fundamentals of building, selecting, validating, searching and refining a surrogate are presented in a manner accessible to novices in the field. Figures are used liberally to explain the key concepts and clearly show the differences between the various techniques, as well as to emphasize the intuitive nature of the conceptual and mathematical reasoning behind them.

More advanced and recent concepts are each presented in stand-alone chapters, allowing the reader to concentrate on material pertinent to their current design problem, and concepts are clearly demonstrated using simple design problems. This collection of advanced concepts (visualization, constraint handling, coping with noisy data, gradient-enhanced modelling, multi-fidelity analysis and multiple objectives) represents an invaluable reference manual for engineers and researchers active in the area.

Engineering Design via Surrogate Modelling is complemented by a suite of Matlab codes, allowing the reader to apply all the techniques presented to their own design problems. By applying statistical modelling to engineering design, this book bridges the wide gap between the engineering and statistics communities. It will appeal to postgraduates and researchers across the academic engineering design community as well as practising design engineers.

  • Provides an inclusive and practical guide to using surrogates in engineering design.
  • Presents the fundamentals of building, selecting, validating, searching and refining a surrogate model.
  • Guides the reader through the practical implementation of a surrogate-based design process using a set of case studies from real engineering design challenges.

Accompanied by a companion website featuring Matlab software at


About the Authors.



Part I: Fundamentals.

1. Sampling Plans.

1.1 The ‘Curse of Dimensionality’ and How to Avoid It.

1.2 Physical versus Computational Experiments.

1.3 Designing Preliminary Experiments (Screening). 

1.3.1 Estimating the Distribution of Elementary Effects.

1.4 Designing a Sampling Plan.

1.4.1 Stratification.

1.4.2 Latin Squares and Random Latin Hypercubes.

1.4.3 Space-filling Latin Hypercubes.

1.4.4 Space-filling Subsets.

1.5 A Note on Harmonic Responses.

1.6 Some Pointers for Further Reading.


2. Constructing a Surrogate.

2.1 The Modelling Process.

2.1.1 Stage One: Preparing the Data and Choosing a Modelling Approach.

2.1.2 Stage Two: Parameter Estimation and Training.

2.1.3 Stage Three: Model Testing.

2.2 Polynomial Models.

2.2.1 Example One: Aerofoil Drag.

2.2.2 Example Two: a Multimodal Testcase.

2.2.3 What About the k-variable Case?

2.3 Radial Basis Function Models.

2.3.1 Fitting Noise-Free Data.

2.3.2 Radial Basis Function Models of Noisy Data.

2.4 Kriging.

2.4.1 Building the Kriging Model.

2.4.2 Kriging Prediction.

2.5 Support Vector Regression.

2.5.1 The Support Vector Predictor.

2.5.2 The Kernel Trick.

2.5.3 Finding the Support Vectors.

2.5.4 Finding µ.

2.5.5 Choosing C and ε.

2.5.6 Computing ε: v-SVR 71.

2.6 The Big(ger) Picture.


3. Exploring and Exploiting a Surrogate.

3.1 Searching the Surrogate.

3.2 Infill Criteria.

3.2.1 Prediction Based Exploitation.

3.2.2 Error Based Exploration.

3.2.3 Balanced Exploitation and Exploration.

3.2.4 Conditional Likelihood Approaches.

3.2.5 Other Methods.

3.3 Managing a Surrogate Based Optimization Process.

3.3.1 Which Surrogate for What Use?

3.3.2 How Many Sample Plan and Infill Points?

3.3.3 Convergence Criteria.

3.3.4 Search of the Vibration Isolator Geometry Feasibility Using Kriging Goal Seeking.


Part II: Advanced Concepts.

4. Visualization.

4.1 Matrices of Contour Plots.

4.2 Nested Dimensions.


5. Constraints.

5.1 Satisfaction of Constraints by Construction.

5.2 Penalty Functions.

5.3 Example Constrained Problem.

5.3.1 Using a Kriging Model of the Constraint Function.

5.3.2 Using a Kriging Model of the Objective Function.

5.4 Expected Improvement Based Approaches.

5.4.1 Expected Improvement With Simple Penalty Function.

5.4.2 Constrained Expected Improvement.

5.5 Missing Data.

5.5.1 Imputing Data for Infeasible Designs.

5.6 Design of a Helical Compression Spring Using Constrained Expected Improvement.

5.7 Summary.


6. Infill Criteria With Noisy Data.

6.1 Regressing Kriging.

6.2 Searching the Regression Model.

6.2.1 Re-Interpolation.

6.2.2 Re-Interpolation With Conditional Likelihood Approaches.

6.3 A Note on Matrix Ill-Conditioning.

6.4 Summary.


7. Exploiting Gradient Information.

7.1 Obtaining Gradients.

7.1.1 Finite Differencing.

7.1.2 Complex Step Approximation.

7.1.3 Adjoint Methods and Algorithmic Differentiation.

7.2 Gradient-enhanced Modelling.

7.3 Hessian-enhanced Modelling.

7.4 Summary.


8. Multi-fidelity Analysis.

8.1 Co-Kriging.

8.2 One-variable Demonstration.

8.3 Choosing Xc and Xe.

8.4 Summary.


9. Multiple Design Objectives.

9.1 Pareto Optimization.

9.2 Multi-objective Expected Improvement.

9.3 Design of the Nowacki Cantilever Beam Using Multi-objective, Constrained Expected Improvement.

9.4 Design of a Helical Compression Spring Using Multi-objective, Constrained Expected Improvement.

9.5 Summary.


Appendix: Example Problems.

A.1 One-Variable Test Function.

A.2 Branin Test Function.

A.3 Aerofoil Design.

A.4 The Nowacki Beam.

A.5 Multi-objective, Constrained Optimal Design of a Helical Compression Spring.

A.6 Novel Passive Vibration Isolator Feasibility.