# How benign is benign overfitting?

arXiv: https://arxiv.org/abs/2007.04028 · HTML: https://mlsanity.com/paper/2007.04028/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2020.

## Built on by (most cited first, 30 of 40)

- [Robustness and Accuracy Could Be Reconcilable by (Proper) Definition](https://mlsanity.com/paper/2202.10103/) (2022)
- [Exploring Memorization in Adversarial Training](https://mlsanity.com/paper/2106.01606/) (2021)
- [Corrective Machine Unlearning](https://mlsanity.com/paper/2402.14015/) (2024)
- [Removing Batch Normalization Boosts Adversarial Training](https://mlsanity.com/paper/2207.01156/) (2022)
- [On the benefits of knowledge distillation for adversarial robustness](https://mlsanity.com/paper/2203.07159/) (2022)
- [How unfair is private learning ?](https://mlsanity.com/paper/2206.03985/) (2022)
- [Interpolation can hurt robust generalization even when there is no noise](https://mlsanity.com/paper/2108.02883/) (2021)
- [Why adversarial training can hurt robust accuracy](https://mlsanity.com/paper/2203.02006/) (2022)
- [Potion: Towards Poison Unlearning](https://mlsanity.com/paper/2406.09173/) (2024)
- [The Curious Case of Benign Memorization](https://mlsanity.com/paper/2210.14019/) (2022)
- [On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training](https://mlsanity.com/paper/2112.07324/) (2021)
- [Benign Overfitting in Adversarially Robust Linear Classification](https://mlsanity.com/paper/2112.15250/) (2021)
- [On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes](https://mlsanity.com/paper/2203.11864/) (2022)
- [Optimal Rate of Kernel Regression in Large Dimensions](https://mlsanity.com/paper/2309.04268/) (2023)
- [Strong inductive biases provably prevent harmless interpolation](https://mlsanity.com/paper/2301.07605/) (2023)
- [Outlier Robust Adversarial Training](https://mlsanity.com/paper/2309.05145/) (2023)
- [It Is All About Data: A Survey on the Effects of Data on Adversarial Robustness](https://mlsanity.com/paper/2303.09767/) (2023)
- [Accuracy on the wrong line: On the pitfalls of noisy data for out-of-distribution generalisation](https://mlsanity.com/paper/2406.19049/) (2024)
- [NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labels](https://mlsanity.com/paper/2105.14676/) (2021)
- [A law of adversarial risk, interpolation, and label noise](https://mlsanity.com/paper/2207.03933/) (2022)
- [One-vs-the-Rest Loss to Focus on Important Samples in Adversarial Training](https://mlsanity.com/paper/2207.10283/) (2022)
- [Learning Sample Reweighting for Accuracy and Adversarial Robustness](https://mlsanity.com/paper/2210.11513/) (2022)
- [Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyond](https://mlsanity.com/paper/2503.07501/) (2025)
- [Multiplicative Reweighting for Robust Neural Network Optimization](https://mlsanity.com/paper/2102.12192/) (2021)
- [Adversarial Training with Complementary Labels: On the Benefit of Gradually Informative Attacks](https://mlsanity.com/paper/2211.00269/) (2022)
- [Combating Exacerbated Heterogeneity for Robust Models in Federated Learning](https://mlsanity.com/paper/2303.00250/) (2023)
- [Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation](https://mlsanity.com/paper/2311.05379/) (2023)
- [How Does Label Noise Gradient Descent Improve Generalization in the Low SNR Regime?](https://mlsanity.com/paper/2510.17526/) (2025)
- [Towards the Memorization Effect of Neural Networks in Adversarial Training](https://mlsanity.com/paper/2106.04794/) (2021)
- [Bridged Adversarial Training](https://mlsanity.com/paper/2108.11135/) (2021)

## Abstract (arXiv)

We investigate two causes for adversarial vulnerability in deep neural networks: bad data and (poorly) trained models. When trained with SGD, deep neural networks essentially achieve zero training error, even in the presence of label noise, while also exhibiting good generalization on natural test data, something referred to as benign overfitting [2, 10]. However, these models are vulnerable to adversarial attacks. We identify label noise as one of the causes for adversarial vulnerability, and provide theoretical and empirical evidence in support of this. Surprisingly, we find several instances of label noise in datasets such as MNIST and CIFAR, and that robustly trained models incur training error on some of these, i.e. they don't fit the noise. However, removing noisy labels alone does not suffice to achieve adversarial robustness. Standard training procedures bias neural networks towards learning "simple" classification boundaries, which may be less robust than more complex ones. We observe that adversarial training does produce more complex decision boundaries. We conjecture that in part the need for complex decision boundaries arises from sub-optimal representation learning. By means of simple toy examples, we show theoretically how the choice of representation can drastically affect adversarial robustness.