# Distribution-Free Uncertainty Quantification for Kernel Methods by Gradient Perturbations

arXiv: https://arxiv.org/abs/1812.09632 · HTML: https://mlsanity.com/paper/1812.09632/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2018.

## Built on by (most cited first, 9 of 9)

- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Nonparametric, Nonasymptotic Confidence Bands with Paley-Wiener Kernels for Band-Limited Functions](https://mlsanity.com/paper/2206.13629/) (2022)
- [Improving Kernel-Based Nonasymptotic Simultaneous Confidence Bands](https://mlsanity.com/paper/2401.15791/) (2024)
- [Sample Complexity of the Sign-Perturbed Sums Identification Method: Scalar Case](https://mlsanity.com/paper/2401.15792/) (2024)
- [Fluids You Can Trust: Property-Preserving Operator Learning for Incompressible Flows](https://mlsanity.com/paper/2602.15472/) (2026)
- [Pointwise confidence estimation in the non-linear ℓ2-regularized least squares](https://mlsanity.com/paper/2506.07088/) (2025)
- [Derandomizing Simultaneous Confidence Regions for Band-Limited Functions by Improved Norm Bounds and Majority-Voting Schemes](https://mlsanity.com/paper/2506.17764/) (2025)
- [Mesters'eges Intelligencia Kutat'asok Magyarorsz'agon](https://mlsanity.com/paper/2503.05767/) (2025)
- [MiNCE: Nonparametric, Strongly Consistent Confidence Envelopes for Band-Limited Functions and their Smoothed Spectra](https://mlsanity.com/paper/2609.09436/) (2026)

## Abstract (arXiv)

We propose a data-driven approach to quantify the uncertainty of models constructed by kernel methods. Our approach minimizes the needed distributional assumptions, hence, instead of working with, for example, Gaussian processes or exponential families, it only requires knowledge about some mild regularity of the measurement noise, such as it is being symmetric or exchangeable. We show, by building on recent results from finite-sample system identification, that by perturbing the residuals in the gradient of the objective function, information can be extracted about the amount of uncertainty our model has. Particularly, we provide an algorithm to build exact, non-asymptotically guaranteed, distribution-free confidence regions for ideal, noise-free representations of the function we try to estimate. For the typical convex quadratic problems and symmetric noises, the regions are star convex centered around a given nominal estimate, and have efficient ellipsoidal outer approximations. Finally, we illustrate the ideas on typical kernel methods, such as LS-SVC, KRR, varepsilon-SVR and kernelized LASSO.