# Practical and Rigorous Uncertainty Bounds for Gaussian Process Regression

arXiv: https://arxiv.org/abs/2105.02796 · HTML: https://mlsanity.com/paper/2105.02796/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 39 references that are in the map.
- First posted on arXiv 6 May 2021.

## Builds on (most cited first, 17 of 17)

- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [On Kernelized Multi-armed Bandits](https://mlsanity.com/paper/1704.00445/) (2017)
- [Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences](https://mlsanity.com/paper/1807.02582/) (2018)
- [High Dimensional Bayesian Optimisation and Bandits via Additive Models](https://mlsanity.com/paper/1503.01673/) (2015)
- [Learning-based Model Predictive Control for Safe Exploration](https://mlsanity.com/paper/1803.08287/) (2018)
- [Safe Controller Optimization for Quadrotors with Gaussian Processes](https://mlsanity.com/paper/1509.01066/) (2015)
- [Uniform Error Bounds for Gaussian Process Regression with Application to Safe Control](https://mlsanity.com/paper/1906.01376/) (2019)
- [Linearly constrained Gaussian processes](https://mlsanity.com/paper/1703.00787/) (2017)
- [Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processes](https://mlsanity.com/paper/1603.04915/) (2016)
- [Scalable Log Determinants for Gaussian Process Kernel Learning](https://mlsanity.com/paper/1711.03481/) (2017)
- [Deterministic error bounds for kernel-based learning techniques under bounded noise](https://mlsanity.com/paper/2008.04005/) (2020)
- [Algorithmic Linearly Constrained Gaussian Processes](https://mlsanity.com/paper/1801.09197/) (2018)
- [Large-scale Log-determinant Computation through Stochastic Chebyshev Expansions](https://mlsanity.com/paper/1503.06394/) (2015)
- [Learning Constrained Dynamics with Gauss Principle adhering Gaussian Processes](https://mlsanity.com/paper/2004.11238/) (2020)
- [Provably Robust Learning-Based Approach for High-Accuracy Tracking Control of Lagrangian Systems](https://mlsanity.com/paper/1804.01031/) (2018)
- [Mean Square Prediction Error of Misspecified Gaussian Process Models](https://mlsanity.com/paper/1811.06642/) (2018)
- [Scalable Gaussian Process Inference with Finite-data Mean and Variance Guarantees](https://mlsanity.com/paper/1806.10234/) (2018)

## Built on by (most cited first, 30 of 49)

- [Gaussian Process Uniform Error Bounds with Unknown Hyperparameters for Safety-Critical Applications](https://mlsanity.com/paper/2109.02606/) (2021)
- [GoSafeOpt: Scalable Safe Exploration for Global Optimization of Dynamical Systems](https://mlsanity.com/paper/2201.09562/) (2022)
- [Bayesian optimization as a flexible and efficient design framework for sustainable process systems](https://mlsanity.com/paper/2401.16373/) (2024)
- [On Safety in Safe Bayesian Optimization](https://mlsanity.com/paper/2403.12948/) (2024)
- [Guaranteed Coverage Prediction Intervals with Gaussian Process Regression](https://mlsanity.com/paper/2310.15641/) (2023)
- [Automatic nonlinear MPC approximation with closed-loop guarantees](https://mlsanity.com/paper/2312.10199/) (2023)
- [Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment](https://mlsanity.com/paper/2109.11679/) (2021)
- [Safe exploration in reproducing kernel Hilbert spaces](https://mlsanity.com/paper/2503.10352/) (2025)
- [Information-Theoretic Safe Exploration with Gaussian Processes](https://mlsanity.com/paper/2212.04914/) (2022)
- [Tuning Legged Locomotion Controllers via Safe Bayesian Optimization](https://mlsanity.com/paper/2306.07092/) (2023)
- [PACSBO: Probably approximately correct safe Bayesian optimization](https://mlsanity.com/paper/2409.01163/) (2024)
- [Safe and Stable Closed-Loop Learning for Neural-Network-Supported Model Predictive Control](https://mlsanity.com/paper/2409.10171/) (2024)
- [BO-Muse: A human expert and AI teaming framework for accelerated experimental design](https://mlsanity.com/paper/2303.01684/) (2023)
- [Towards Safe Multi-Task Bayesian Optimization](https://mlsanity.com/paper/2312.07281/) (2023)
- [Towards safe Bayesian optimization with Wiener kernel regression](https://mlsanity.com/paper/2411.02253/) (2024)
- [On Foundation Models for Dynamical Systems from Purely Synthetic Data](https://mlsanity.com/paper/2412.00395/) (2024)
- [The Hardness of Validating Observational Studies with Experimental Data](https://mlsanity.com/paper/2503.14795/) (2025)
- [Optimal kernel regression bounds under energy-bounded noise](https://mlsanity.com/paper/2505.22235/) (2025)
- [Information-theoretic Analysis of Test Data Sensitivity in Uncertainty](https://mlsanity.com/paper/2307.12456/) (2023)
- [Robust Bayesian Satisficing](https://mlsanity.com/paper/2308.08291/) (2023)
- [Exact Inference for Continuous-Time Gaussian Process Dynamics](https://mlsanity.com/paper/2309.02351/) (2023)
- [Lipschitz and H"older Continuity in Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/2310.18078/) (2023)
- [Data-Driven Abstractions via Binary-Tree Gaussian Processes for Formal Verification](https://mlsanity.com/paper/2407.21029/) (2024)
- [Learning-enhanced robust controller synthesis with rigorous statistical and control-theoretic guarantees](https://mlsanity.com/paper/2105.03397/) (2021)
- [Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions](https://mlsanity.com/paper/2208.10733/) (2022)
- [Meta-Analysis with Untrusted Data](https://mlsanity.com/paper/2407.09387/) (2024)
- [A Hierarchical Surrogate Model for Efficient Multi-Task Parameter Learning in Closed-Loop Control](https://mlsanity.com/paper/2508.12738/) (2025)
- [Uncertainty-aware transfer across tasks using hybrid model-based successor feature reinforcement learning](https://mlsanity.com/paper/2310.10818/) (2023)
- [Safe reinforcement learning in uncertain contexts](https://mlsanity.com/paper/2401.05876/) (2024)
- [An Analysis of Safety Guarantees in Multi-Task Bayesian Optimization](https://mlsanity.com/paper/2503.08555/) (2025)

## Abstract (arXiv)

Gaussian Process Regression is a popular nonparametric regression method based on Bayesian principles that provides uncertainty estimates for its predictions. However, these estimates are of a Bayesian nature, whereas for some important applications, like learning-based control with safety guarantees, frequentist uncertainty bounds are required. Although such rigorous bounds are available for Gaussian Processes, they are too conservative to be useful in applications. This often leads practitioners to replacing these bounds by heuristics, thus breaking all theoretical guarantees. To address this problem, we introduce new uncertainty bounds that are rigorous, yet practically useful at the same time. In particular, the bounds can be explicitly evaluated and are much less conservative than state of the art results. Furthermore, we show that certain model misspecifications lead to only graceful degradation. We demonstrate these advantages and the usefulness of our results for learning-based control with numerical examples.