# Global quantitative robustness of regression feed-forward neural networks

arXiv: https://arxiv.org/abs/2211.10124 · HTML: https://mlsanity.com/paper/2211.10124/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 59 references that are in the map.
- First posted on arXiv 18 Nov 2022.

## Builds on (most cited first, 16 of 16)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems](https://mlsanity.com/paper/1903.12394/) (2019)
- [Two models of double descent for weak features](https://mlsanity.com/paper/1903.07571/) (2019)
- [Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data](https://mlsanity.com/paper/2202.05928/) (2022)
- [The Modern Mathematics of Deep Learning](https://mlsanity.com/paper/2105.04026/) (2021)
- [Trustworthy AI](https://mlsanity.com/paper/2011.02272/) (2020)
- [Reliable Graph Neural Networks via Robust Aggregation](https://mlsanity.com/paper/2010.15651/) (2020)
- [Understanding Structural Vulnerability in Graph Convolutional Networks](https://mlsanity.com/paper/2108.06280/) (2021)
- [Robust Optimization for Deep Regression](https://mlsanity.com/paper/1505.06606/) (2015)
- [On Correctness of Automatic Differentiation for Non-Differentiable Functions](https://mlsanity.com/paper/2006.06903/) (2020)
- [Model Comparison and Calibration Assessment: User Guide for Consistent Scoring Functions in Machine Learning and Actuarial Practice](https://mlsanity.com/paper/2202.12780/) (2022)
- [DeepMoM: Robust Deep Learning With Median-of-Means](https://mlsanity.com/paper/2105.14035/) (2021)

## Abstract (arXiv)

Neural networks are an indispensable model class for many complex learning tasks. Despite the popularity and importance of neural networks and many different established techniques from literature for stabilization and robustification of the training, the classical concepts from robust statistics have rarely been considered so far in the context of neural networks. Therefore, we adapt the notion of the regression breakdown point to regression neural networks and compute the breakdown point for different feed-forward network configurations and contamination settings. In an extensive simulation study, we compare the performance, measured by the out-of-sample loss, by a proxy of the breakdown rate and by the training steps, of non-robust and robust regression feed-forward neural networks in a plethora of different configurations. The results indeed motivate to use robust loss functions for neural network training.