Sampling from Gaussian Processes: A Tutorial and Applications in Global Sensitivity Analysis and Optimization
Not on the map. Why?
Abstract (arXiv)
High-fidelity simulations and physical experiments are essential for engineering analysis and design, yet their high cost often makes two critical tasks--global sensitivity analysis (GSA) and optimization--prohibitively expensive. This limitation motivates the common use of Gaussian processes (GPs) as proxy regression models that provide uncertainty-aware predictions from a limited number of high-quality observations. GPs naturally enable efficient sampling strategies that support informed decision-making under uncertainty by extracting information from a subset of possible functions for the model of interest. However, direct sampling from GPs is inefficient due to their infinite-dimensional nature and the high cost associated with large covariance matrix operations. Despite their popularity in machine learning and statistics communities, sampling from GPs has received little attention in the community of engineering optimization. In this paper, we present the formulation and detailed implementation of two notable sampling methods--random Fourier features and pathwise conditioning--for generating posterior samples from GPs at reduced computational cost. Alternative approaches are briefly described. Importantly, we detail how the generated samples can be applied in GSA, single-objective optimization, and multi-objective optimization. We show successful applications of these sampling methods through a series of numerical examples.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 45 of its 143 references that are in the map. First posted on arXiv 19 Jul 2025.
Builds on
45 of 143 references.
References · 45
- 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
- 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 609
- 2017A Tutorial on Thompson Samplingcited by 433
- 2013Gaussian Processes for Big Datacited by 383
- 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
- 2017On Kernelized Multi-armed Banditscited by 352
- 2013Learning to Optimize Via Posterior Samplingcited by 256
- 2017Max-value Entropy Search for Efficient Bayesian Optimizationcited by 206
- 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 204
- 2020pymoo: Multi-objective Optimization in Pythoncited by 200
- 2020Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimizationcited by 179
- 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 149
33 more
- 2016GPflow: A Gaussian process library using TensorFlowcited by 140
- 2017Machine Learning of Linear Differential Equations using Gaussian Processescited by 140
- 2021Solving and Learning Nonlinear PDEs with Gaussian Processescited by 129
- 2020Efficiently Sampling Functions from Gaussian Process Posteriorscited by 127
- 2018Maximizing acquisition functions for Bayesian optimizationcited by 114
- 2014Hilbert Space Methods for Reduced-Rank Gaussian Process Regressioncited by 88
- 2021Multi-Objective Bayesian Optimization over High-Dimensional Search Spacescited by 87
- 2015On the Error of Random Fourier Featurescited by 76
- 2017Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Spacecited by 76
- 2017Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processescited by 66
- 2020Pathwise Conditioning of Gaussian Processescited by 64
- 2019Multi-objective Bayesian Optimization using Pareto-frontier Entropycited by 57
- 2022Joint Entropy Search for Maximally-Informed Bayesian Optimizationcited by 55
- 2016Parallel Bayesian Global Optimization of Expensive Functionscited by 44
- 2014Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernelscited by 42
- 2021Bayesian Optimization with High-Dimensional Outputscited by 32
- 2015Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functionscited by 31
- 2020Practical Hilbert space approximate Bayesian Gaussian processes for probabilistic programmingcited by 28
- 2021Latent Map Gaussian Processes for Mixed Variable Metamodelingcited by 23
- 2016Gaussian process emulators for computer experiments with inequality constraintscited by 17
- 2023GP+: A Python Library for Kernel-based learning via Gaussian Processescited by 17
- 2020Scaled Vecchia approximation for fast computer-model emulationcited by 15
- 2020The Hypervolume Indicator: Problems and Algorithmscited by 12
- 2023Multi-fidelity Bayesian Optimization: A Reviewcited by 12
- 2020Nonnegativity-Enforced Gaussian Process Regressioncited by 10
- 2024Epsilon-Greedy Thompson Sampling to Bayesian Optimizationcited by 9
- 2024Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processescited by 5
- 2024Optimizing Posterior Samples for Bayesian Optimization via Rootfindingcited by 5
- 2022Orthonormal Expansions for Translation-Invariant Kernelscited by 3
- 2024Automated design of nonreciprocal thermal emitters via Bayesian optimizationcited by 3
- 2024Multi-fidelity Machine Learning for Uncertainty Quantification and Optimizationcited by 3
- 2023On Active Learning for Gaussian Process-based Global Sensitivity Analysiscited by 2
- 2024Gaussian Process Thompson Sampling via Rootfindingcited by 2
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.