# On Connections between Regularizations for Improving DNN Robustness

arXiv: https://arxiv.org/abs/2007.02209 · HTML: https://mlsanity.com/paper/2007.02209/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jul 2020.

## Built on by (most cited first, 5 of 5)

- [It Is All About Data: A Survey on the Effects of Data on Adversarial Robustness](https://mlsanity.com/paper/2303.09767/) (2023)
- [Recent Advances in Large Margin Learning](https://mlsanity.com/paper/2103.13598/) (2021)
- [Bridging the Gap Between Adversarial Robustness and Optimization Bias](https://mlsanity.com/paper/2102.08868/) (2021)
- [Fast Multi-grid Methods for Minimizing Curvature Energy](https://mlsanity.com/paper/2204.07921/) (2022)
- [Asymptotic Behavior of Adversarial Training Estimator under ℓ_∞-Perturbation](https://mlsanity.com/paper/2401.15262/) (2024)

## Abstract (arXiv)

This paper analyzes regularization terms proposed recently for improving the adversarial robustness of deep neural networks (DNNs), from a theoretical point of view. Specifically, we study possible connections between several effective methods, including input-gradient regularization, Jacobian regularization, curvature regularization, and a cross-Lipschitz functional. We investigate them on DNNs with general rectified linear activations, which constitute one of the most prevalent families of models for image classification and a host of other machine learning applications. We shed light on essential ingredients of these regularizations and re-interpret their functionality. Through the lens of our study, more principled and efficient regularizations can possibly be invented in the near future.