# Practical heteroskedastic Gaussian process modeling for large simulation experiments

arXiv: https://arxiv.org/abs/1611.05902 · HTML: https://mlsanity.com/paper/1611.05902/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2016.

## Built on by (most cited first, 30 of 39)

- [A review of predictive uncertainty estimation with machine learning](https://mlsanity.com/paper/2209.08307/) (2022)
- [Risk-averse Heteroscedastic Bayesian Optimization](https://mlsanity.com/paper/2111.03637/) (2021)
- [Triangulation candidates for Bayesian optimization](https://mlsanity.com/paper/2112.07457/) (2021)
- [Modeling the Machine Learning Multiverse](https://mlsanity.com/paper/2206.05985/) (2022)
- [A portfolio approach to massively parallel Bayesian optimization](https://mlsanity.com/paper/2110.09334/) (2021)
- [Active Learning of Piecewise Gaussian Process Surrogates](https://mlsanity.com/paper/2301.08789/) (2023)
- [Robust expected improvement for Bayesian optimization](https://mlsanity.com/paper/2302.08612/) (2023)
- [Sensitivity Prewarping for Local Surrogate Modeling](https://mlsanity.com/paper/2101.06296/) (2021)
- [Bayesian Calibration for Activity Based Models](https://mlsanity.com/paper/2203.04414/) (2022)
- [Polynomial Chaos Surrogate Construction for Random Fields with Parametric Uncertainty](https://mlsanity.com/paper/2311.00553/) (2023)
- [All Emulators are Wrong, Many are Useful, and Some are More Useful Than Others: A Reproducible Comparison of Computer Model Surrogates](https://mlsanity.com/paper/2512.09060/) (2025)
- [Trajectory-oriented optimization of stochastic epidemiological models](https://mlsanity.com/paper/2305.03926/) (2023)
- [Multi-Response Heteroscedastic Gaussian Process Models and Their Inference](https://mlsanity.com/paper/2308.15370/) (2023)
- [Reliability analysis for non-deterministic limit-states using stochastic emulators](https://mlsanity.com/paper/2412.13731/) (2024)
- [Mixed Likelihood Variational Gaussian Processes](https://mlsanity.com/paper/2503.04138/) (2025)
- [Task-Adaptive Robot Learning from Demonstration with Gaussian Process Models under Replication](https://mlsanity.com/paper/2010.07795/) (2020)
- [Goal-oriented adaptive sampling under random field modelling of response probability distributions](https://mlsanity.com/paper/2102.07612/) (2021)
- [Facilitating automated conversion of scientific knowledge into scientific simulation models with the Machine Assisted Generation, Calibration, and Comparison (MAGCC) Framework](https://mlsanity.com/paper/2204.10382/) (2022)
- [Bayesian multi-objective optimization for stochastic simulators: an extension of the Pareto Active Learning method](https://mlsanity.com/paper/2207.03842/) (2022)
- [Stochastic Cell Transmission Models of Traffic Networks](https://mlsanity.com/paper/2304.11654/) (2023)
- [Achieving Diversity in Objective Space for Sample-efficient Search of Multiobjective Optimization Problems](https://mlsanity.com/paper/2306.13780/) (2023)
- [Gearing Gaussian process modeling and sequential design towards stochastic simulators](https://mlsanity.com/paper/2412.07306/) (2024)
- [Scalable and adaptive prediction bands with kernel sum-of-squares](https://mlsanity.com/paper/2505.21039/) (2025)
- [AL-SPCE - Reliability analysis for nondeterministic models using stochastic polynomial chaos expansions and active learning](https://mlsanity.com/paper/2507.04553/) (2025)
- [Scaling Bayesian Optimization With Game Theory](https://mlsanity.com/paper/2110.03790/) (2021)
- [Sensitivity Analysis in the Presence of Intrinsic Stochasticity for Discrete Fracture Network Simulations](https://mlsanity.com/paper/2312.04722/) (2023)
- [Quantifying Local Model Validity using Active Learning](https://mlsanity.com/paper/2406.07474/) (2024)
- [Multifidelity Cross-validation](https://mlsanity.com/paper/2407.01495/) (2024)
- [Adaptive Replication Strategies in Trust-Region-Based Bayesian Optimization of Stochastic Functions](https://mlsanity.com/paper/2504.20527/) (2025)
- [Adaptive Linear Embedding for Nonstationary High-Dimensional Optimization](https://mlsanity.com/paper/2505.11281/) (2025)

## Abstract (arXiv)

We present a unified view of likelihood based Gaussian progress regression for simulation experiments exhibiting input-dependent noise. Replication plays an important role in that context, however previous methods leveraging replicates have either ignored the computational savings that come from such design, or have short-cut full likelihood-based inference to remain tractable. Starting with homoskedastic processes, we show how multiple applications of a well-known Woodbury identity facilitate inference for all parameters under the likelihood (without approximation), bypassing the typical full-data sized calculations. We then borrow a latent-variable idea from machine learning to address heteroskedasticity, adapting it to work within the same thrifty inferential framework, thereby simultaneously leveraging the computational and statistical efficiency of designs with replication. The result is an inferential scheme that can be characterized as single objective function, complete with closed form derivatives, for rapid library-based optimization. Illustrations are provided, including real-world simulation experiments from manufacturing and the management of epidemics.