# Double descent in the condition number

arXiv: https://arxiv.org/abs/1912.06190 · HTML: https://mlsanity.com/paper/1912.06190/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2019.

## Built on by (most cited first, 14 of 14)

- [Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle](https://mlsanity.com/paper/2303.14151/) (2023)
- [A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning](https://mlsanity.com/paper/2310.18988/) (2023)
- [Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations](https://mlsanity.com/paper/2406.09366/) (2024)
- [Distilling Double Descent](https://mlsanity.com/paper/2102.06849/) (2021)
- [On the Role of Optimization in Double Descent: A Least Squares Study](https://mlsanity.com/paper/2107.12685/) (2021)
- [Conditioning of Random Feature Matrices: Double Descent and Generalization Error](https://mlsanity.com/paper/2110.11477/) (2021)
- [On the Double Descent of Random Features Models Trained with SGD](https://mlsanity.com/paper/2110.06910/) (2021)
- [Log-Linear-Time Gaussian Processes Using Binary Tree Kernels](https://mlsanity.com/paper/2210.01633/) (2022)
- [Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarks](https://mlsanity.com/paper/2502.18339/) (2025)
- [Bandwidth Selection for Gaussian Kernel Ridge Regression via Jacobian Control](https://mlsanity.com/paper/2205.11956/) (2022)
- [Changing the Kernel During Training Leads to Double Descent in Kernel Regression](https://mlsanity.com/paper/2311.01762/) (2023)
- [Probability Distribution Learning and Its Application in Deep Learning](https://mlsanity.com/paper/2406.05666/) (2024)
- [Understanding the Role of Optimization in Double Descent](https://mlsanity.com/paper/2312.03951/) (2023)
- [A Rigorous, Tractable Measure of Model Complexity](https://mlsanity.com/paper/2605.21167/) (2026)

## Abstract (arXiv)

In solving a system of n linear equations in d variables Ax=b, the condition number of the n,d matrix A measures how much errors in the data b affect the solution x. Estimates of this type are important in many inverse problems. An example is machine learning where the key task is to estimate an underlying function from a set of measurements at random points in a high dimensional space and where low sensitivity to error in the data is a requirement for good predictive performance. Here we discuss the simple observation, which is known but surprisingly little quoted (see Theorem 4.2 in citeBrgisser:2013:CGN:2526261): when the columns of A are random vectors, the condition number of A is highest if d=n, that is when the inverse of A exists. An overdetermined system (n>d) as well as an underdetermined system (n<d), for which the pseudoinverse must be used instead of the inverse, typically have significantly better, that is lower, condition numbers. Thus the condition number of A plotted as function of d shows a double descent behavior with a peak at d=n.