# Bayesian Double Descent

arXiv: https://arxiv.org/abs/2507.07338 · HTML: https://mlsanity.com/paper/2507.07338/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 43 references that are in the map.
- First posted on arXiv 9 Jul 2025.

## Builds on (most cited first, 7 of 7)

- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [High-dimensional analysis of double descent for linear regression with random projections](https://mlsanity.com/paper/2303.01372/) (2023)
- [Deep Learning: A Bayesian Perspective](https://mlsanity.com/paper/1706.00473/) (2017)
- [Deep Learning for Spatio-Temporal Modeling: Dynamic Traffic Flows and High Frequency Trading](https://mlsanity.com/paper/1705.09851/) (2017)
- [Generative Causal Inference](https://mlsanity.com/paper/2306.16096/) (2023)

## Built on by (most cited first, 1 of 1)

- [Generative Bayesian Hyperparameter Tuning](https://mlsanity.com/paper/2512.20051/) (2025)

## Abstract (arXiv)

Double descent is a phenomenon of over-parameterized statistical models such as deep neural networks which have a re-descending property in their risk function. As the complexity of the model increases, risk exhibits a U-shaped region due to the traditional bias-variance trade-off, then as the number of parameters equals the number of observations and the model becomes one of interpolation where the risk can be unbounded and finally, in the over-parameterized region, it re-descends -- the double descent effect. Our goal is to show that this has a natural Bayesian interpretation. We also show that this is not in conflict with the traditional Occam's razor -- simpler models are preferred to complex ones, all else being equal. Our theoretical foundations use Bayesian model selection, the Dickey-Savage density ratio, and connect generalized ridge regression and global-local shrinkage methods with double descent. We illustrate our approach for high dimensional neural networks and provide detailed treatments of infinite Gaussian means models and non-parametric regression. Finally, we conclude with directions for future research.