mlsanityData to 7 Oct 2026

All Emulators are Wrong, Many are Useful, and Some are More Useful Than Others: A Reproducible Comparison of Computer Model Surrogates

arXiv 2512.09060 ↗First posted 9 Dec 2025stat.CO stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

Accurate and efficient surrogate modeling is essential for modern computational science, and there are a staggering number of emulation methods to choose from. With new methods being developed all the time, comparing the relative strengths and weaknesses of different methods remains a challenge due to inconsistent benchmarking practices and (sometimes) limited reproducibility and transparency. In this work, we present a large-scale, fully reproducible comparison of 29 distinct emulators across 60 canonical test functions and 40 real emulation datasets. To facilitate rigorous, apples-to-apples comparisons, we introduce the R package textttduqling, which streamlines reproducible simulation studies using a consistent, simple syntax, and automatic internal scaling of inputs. This framework allows researchers to compare emulators in a unified environment and makes it possible to replicate or extend previous studies with minimal effort, even across different publications. Our results provide detailed empirical insight into the strengths and weaknesses of state-of-the-art emulators and offer guidance for both method developers and practitioners selecting a surrogate for new data. We discuss best practices for emulator comparison and highlight how textttduqling can accelerate research in emulator design and application.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 40 of its 161 references that are in the map. First posted on arXiv 9 Dec 2025.

Builds on

40 of 161 references.

References · 40

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  2. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  3. 2013Gaussian Processes for Big Datacited by 377
  4. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 364
  5. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  6. 2018When Gaussian Process Meets Big Data: A Review of Scalable GPscited by 205
  7. 2019NGBoost: Natural Gradient Boosting for Probabilistic Predictioncited by 94
  8. 2015Distributed Gaussian Processescited by 84
  9. 2018Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantificationcited by 81
  10. 2022Exploring the Whole Rashomon Set of Sparse Decision Treescited by 80
  11. 2017A general framework for Vecchia approximations of Gaussian processescited by 66
  12. 2013Local Gaussian process approximation for large computer experimentscited by 59
28 more
  1. 2024Amazing Things Come From Having Many Good Modelscited by 52
  2. 2017A Case Study Competition Among Methods for Analyzing Large Spatial Datacited by 44
  3. 2023JANA: Jointly Amortized Neural Approximation of Complex Bayesian Modelscited by 41
  4. 2016Practical heteroskedastic Gaussian process modeling for large simulation experimentscited by 38
  5. 2020Active Learning for Deep Gaussian Process Surrogatescited by 38
  6. 2015Polynomial-Chaos-based Krigingcited by 21
  7. 2018Vecchia approximations of Gaussian-process predictionscited by 21
  8. 2018Extending classical surrogate modelling to high-dimensions through supervised dimensionality reduction: a data-driven approachcited by 16
  9. 2019Sequential Learning of Active Subspacescited by 16
  10. 2020Scaled Vecchia approximation for fast computer-model emulationcited by 15
  11. 2023Nonstationary Gaussian Process Surrogatescited by 8
  12. 2022GP-BART: a novel Bayesian additive regression trees approach using Gaussian processescited by 7
  13. 2022A spectral surrogate model for stochastic simulators computed from trajectory samplescited by 7
  14. 2022PCENet: High Dimensional Surrogate Modeling for Learning Uncertaintycited by 5
  15. 2023Model Mixing Using Bayesian Additive Regression Treescited by 4
  16. 2023Co-Active Subspace Methods for the Joint Analysis of Adjacent Computer Modelscited by 4
  17. 2023Discovering Active Subspaces for High-Dimensional Computer Modelscited by 3
  18. 2020Analyzing Stochastic Computer Models: A Review with Opportunitiescited by 2
  19. 2022Bayesian Projection Pursuit Regressioncited by 2
  20. 2023Elastic Bayesian Model Calibrationcited by 2
  21. 2023Generalized Bayesian MARS: Tools for Emulating Stochastic Computer Modelscited by 2
  22. 2024Fast Emulation, Modular Calibration, and Active Learning for Simulators with Functional Responsecited by 2
  23. 2025Modular Jump Gaussian Processescited by 2
  24. 2015Emulation of multivariate simulators using thin-plate splines with application to atmospheric dispersioncited by 1
  25. 2019Predicting the Output From a Stochastic Computer Model When a Deterministic Approximation is Availablecited by 1
  26. 2023Estimating Shapley Effects in Big-Data Emulation and Regression Settings using Bayesian Additive Regression Treescited by 1
  27. 2024GPTreeO: An R package for continual regression with dividing local Gaussian processescited by 1
  28. 2025Bayesian Adaptive Polynomial Chaos Expansionscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026A Blueprint for Self-Evolving Coding Agents in Vehicle Aerodynamic Drag Predictioncited by 0
  2. 2026GNet: A scalable and flexible Gaussian process network with nonparametric neuronscited by 0
  3. 2026Adaptive multi-resolution Gaussian processes: Scalable exact inference with naturally data-sparse covariance matricescited by 0