# Origins and mitigation of double descent in reduced order modeling

arXiv: https://arxiv.org/abs/2607.26414 · HTML: https://mlsanity.com/paper/2607.26414/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 66 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Builds on (most cited first, 25 of 25)

- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [The generalization error of random features regression: Precise asymptotics and double descent curve](https://mlsanity.com/paper/1908.05355/) (2019)
- [Two models of double descent for weak features](https://mlsanity.com/paper/1903.07571/) (2019)
- [Modern Bayesian Experimental Design](https://mlsanity.com/paper/2302.14545/) (2023)
- [Optimal Regularization Can Mitigate Double Descent](https://mlsanity.com/paper/2003.01897/) (2020)
- [Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime](https://mlsanity.com/paper/2003.01054/) (2020)
- [The Shape of Learning Curves: a Review](https://mlsanity.com/paper/2103.10948/) (2021)
- [A jamming transition from under- to over-parametrization affects loss landscape and generalization](https://mlsanity.com/paper/1810.09665/) (2018)
- [Triple descent and the two kinds of overfitting: Where & why do they appear?](https://mlsanity.com/paper/2006.03509/) (2020)
- [Data-Driven Sparse Sensor Placement for Reconstruction](https://mlsanity.com/paper/1701.07569/) (2017)
- [A Brief Prehistory of Double Descent](https://mlsanity.com/paper/2004.04328/) (2020)
- [Memorizing without overfitting: Bias, variance, and interpolation in over-parameterized models](https://mlsanity.com/paper/2010.13933/) (2020)
- [Reduced Order Modeling with Shallow Recurrent Decoder Networks](https://mlsanity.com/paper/2502.10930/) (2025)
- [Minimizers of the Empirical Risk and Risk Monotonicity](https://mlsanity.com/paper/1907.05476/) (2019)
- [Multi-fidelity sensor selection: Greedy algorithms to place cheap and expensive sensors with cost constraints](https://mlsanity.com/paper/2005.03650/) (2020)
- [PySensors: A Python Package for Sparse Sensor Placement](https://mlsanity.com/paper/2102.13476/) (2021)
- [The Peaking Phenomenon in Semi-supervised Learning](https://mlsanity.com/paper/1610.05160/) (2016)
- [Unveiling Multiple Descents in Unsupervised Autoencoders](https://mlsanity.com/paper/2406.11703/) (2024)
- [Data-Induced Interactions of Sparse Sensors Using Statistical Physics](https://mlsanity.com/paper/2307.11838/) (2023)
- [PySensors 2.0: A Python Package for Sparse Sensor Placement](https://mlsanity.com/paper/2509.08017/) (2025)
- [Exposure theory for learning complex networks with random walks](https://mlsanity.com/paper/2202.11262/) (2022)
- [Learning Dynamic Graphs, Too Slow](https://mlsanity.com/paper/2207.02177/) (2022)
- [State Estimation Using Sparse DEIM and Recurrent Neural Networks](https://mlsanity.com/paper/2410.15982/) (2024)

## Abstract (arXiv)

Latent low-dimensional structure in datasets of natural and engineered systems enables their sparse sensing, or full-state reconstruction from historical data and very few carefully chosen localized measurements. Depending on the reconstruction algorithm, sensor locations, and measurement noise, the reconstruction risk curves demonstrate a diversity of patterns including a dramatic peak in error known as double descent in Machine Learning literature. Here we explore those scenarios under a unified Data-Noise Averaging theory. Qualitatively, we formulate sufficient criteria for double descent to emerge through a catastrophic amplification of a pathological signal in reconstruction. Quantitatively, we predict the detailed risk curves at a fraction of computational cost, trace reconstruction instability to individual sensors and their combinations, and provide regularization mechanisms to mitigate the instability. We demonstrate results for both static reconstruction of Sea Surface Temperature patterns and time integration of a reduced order model of a PDE.