# The Fourth Quadrant: A Stylized View of Benign Misfitting

arXiv: https://arxiv.org/abs/2608.01032 · HTML: https://mlsanity.com/paper/2608.01032/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 222 references that are in the map.
- First posted on arXiv 2 Aug 2026.

## Builds on (most cited first, 25 of 25)

- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [An Empirical Model of Large-Batch Training](https://mlsanity.com/paper/1812.06162/) (2018)
- [Three Factors Influencing Minima in SGD](https://mlsanity.com/paper/1711.04623/) (2017)
- [The large learning rate phase of deep learning: the catapult mechanism](https://mlsanity.com/paper/2003.02218/) (2020)
- [Measuring the Effects of Data Parallelism on Neural Network Training](https://mlsanity.com/paper/1811.03600/) (2018)
- [A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning](https://mlsanity.com/paper/2109.02355/) (2021)
- [Scaling Laws in Linear Regression: Compute, Parameters, and Data](https://mlsanity.com/paper/2406.08466/) (2024)
- [Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey](https://mlsanity.com/paper/1507.01030/) (2015)
- [Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression](https://mlsanity.com/paper/2110.06198/) (2021)
- [Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD](https://mlsanity.com/paper/2412.20553/) (2024)
- [Benign Underfitting of Stochastic Gradient Descent](https://mlsanity.com/paper/2202.13361/) (2022)
- [Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorization](https://mlsanity.com/paper/2402.09327/) (2024)
- [Improved Scaling Laws in Linear Regression via Data Reuse](https://mlsanity.com/paper/2506.08415/) (2025)
- [The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness](https://mlsanity.com/paper/2401.12236/) (2024)
- [Near-Interpolators: Rapid Norm Growth and the Trade-Off between Interpolation and Generalization](https://mlsanity.com/paper/2403.07264/) (2024)
- [The Dimension Strikes Back with Gradients: Generalization of Gradient Methods in Stochastic Convex Optimization](https://mlsanity.com/paper/2401.12058/) (2024)
- [The Sample Complexity of Gradient Descent in Stochastic Convex Optimization](https://mlsanity.com/paper/2404.04931/) (2024)
- [The Price of Implicit Bias in Adversarially Robust Generalization](https://mlsanity.com/paper/2406.04981/) (2024)
- [Rapid Overfitting of Multi-Pass Stochastic Gradient Descent in Stochastic Convex Optimization](https://mlsanity.com/paper/2505.08306/) (2025)
- [Robust Linear Regression: Phase-Transitions and Precise Tradeoffs for General Norms](https://mlsanity.com/paper/2308.00556/) (2023)
- [Membership Inference Attacks Beyond Overfitting](https://mlsanity.com/paper/2511.16792/) (2025)
- [Scaling Laws for Dynamic Mini-Batch SGD in Sketched Linear Regression](https://mlsanity.com/paper/2605.24316/) (2026)
- [Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective](https://mlsanity.com/paper/2109.13215/) (2021)
- [Shrinkage to Infinity: Reducing Test Error by Inflating the Minimum Norm Interpolator in Linear Models](https://mlsanity.com/paper/2510.19206/) (2025)
- [Optimizer Dynamics at the Edge of Stability with Differential Privacy](https://mlsanity.com/paper/2512.19019/) (2025)

## Abstract (arXiv)

Training error is what we can observe on a training set; test error is the quantity we care about. We study squared-error linear regression in a deterministic (d+1)-dimensional single-spike model. Each training vector has spike coordinate √γ, γ>1, and equal-norm, mutually orthogonal nuisance components. Training labels are 1. Fresh test points are centered Gaussian with covariance operatornamediag(γ,1,ldots,1) and noise-free label x[1]/√γ. We compute the exact train--test tradeoff for linear predictors in the training span. When the spike direction is unknown, allowing all linear predictors does not improve this tradeoff, on average or in the worst case over directions consistent with the data. There is a range of training-set sizes where every span predictor that generalizes well must fit the training data worse than the zero predictor. We call this benign misfitting, or the fourth quadrant. At its sharpest, test error tends to zero while training error diverges, with both zero-predictor errors normalized to one. The best span predictor begins to generalize when ngg d/γ2, while interpolation needs ngg d/γ. In the intervening window d/γ2ll nll d/γ, useful span prediction lies beyond interpolation: training predictions overshoot the labels. One-pass stochastic gradient descent with a large constant learning rate attains small test error throughout this window, matching the best span predictor up to a logarithmic factor, while leaving large empirical training error. Finally, the nuisance component responsible for the training misfit also controls adversarial sensitivity.