Asymptotics of Bayesian Uncertainty Estimation in Random Features Regression
Not on the map. Why?
Abstract (arXiv)
In this paper we compare and contrast the behavior of the posterior predictive distribution to the risk of the maximum a posteriori estimator for the random features regression model in the overparameterized regime. We will focus on the variance of the posterior predictive distribution (Bayesian model average) and compare its asymptotics to that of the risk of the MAP estimator. In the regime where the model dimensions grow faster than any constant multiple of the number of samples, asymptotic agreement between these two quantities is governed by the phase transition in the signal-to-noise ratio. They also asymptotically agree with each other when the number of samples grow faster than any constant multiple of model dimensions. Numerical simulations illustrate finer distributional properties of the two quantities for finite dimensions. We conjecture they have Gaussian fluctuations and exhibit similar properties as found by previous authors in a Gaussian sequence model, which is of independent theoretical interest.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 30 references that are in the map. First posted on arXiv 6 Jun 2023.
Builds on
16 of 30 references.
References · 16
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
- 2018On Lazy Training in Differentiable Programmingcited by 675
- 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 513
- 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 466
- 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 413
- 2019Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limitcited by 187
- 2013A General Framework for Updating Belief Distributionscited by 175
- 2019Linearized two-layers neural networks in high dimensioncited by 155
- 2021Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolationcited by 132
- 2021Bayesian Neural Network Priors Revisitedcited by 121
- 2020The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalizationcited by 101
4 more
- 2022Sharp Asymptotics of Kernel Ridge Regression Beyond the Linear Regimecited by 19
- 2022Theoretical characterization of uncertainty in high-dimensional linear classificationcited by 18
- 2022On double-descent in uncertainty quantification in overparametrized modelscited by 16
- 2015Bayesian Approximate Kernel Regression with Variable Selectioncited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.