# Adversarially robust generalization theory via Jacobian regularization for deep neural networks

arXiv: https://arxiv.org/abs/2412.12449 · HTML: https://mlsanity.com/paper/2412.12449/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Builds on (most cited first, 25 of 25)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Adversarially Robust Generalization Requires More Data](https://mlsanity.com/paper/1804.11285/) (2018)
- [A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks](https://mlsanity.com/paper/1707.09564/) (2017)
- [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017)
- [Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples](https://mlsanity.com/paper/2010.03593/) (2020)
- [Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients](https://mlsanity.com/paper/1711.09404/) (2017)
- [Sensitivity and Generalization in Neural Networks: an Empirical Study](https://mlsanity.com/paper/1802.08760/) (2018)
- [A vector-contraction inequality for Rademacher complexities](https://mlsanity.com/paper/1605.00251/) (2016)
- [Rademacher Complexity for Adversarially Robust Generalization](https://mlsanity.com/paper/1810.11914/) (2018)
- [Robust Learning with Jacobian Regularization](https://mlsanity.com/paper/1908.02729/) (2019)
- [Robust Large Margin Deep Neural Networks](https://mlsanity.com/paper/1605.08254/) (2016)
- [Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization](https://mlsanity.com/paper/1803.08680/) (2018)
- [Learning with a Strong Adversary](https://mlsanity.com/paper/1511.03034/) (2015)
- [A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies](https://mlsanity.com/paper/2203.14046/) (2022)
- [A Unified Gradient Regularization Family for Adversarial Examples](https://mlsanity.com/paper/1511.06385/) (2015)
- [Jacobian Adversarially Regularized Networks for Robustness](https://mlsanity.com/paper/1912.10185/) (2019)
- [Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization](https://mlsanity.com/paper/1511.05432/) (2015)
- [Gradient Regularization Improves Accuracy of Discriminative Models](https://mlsanity.com/paper/1712.09936/) (2017)
- [Statistical Guarantees for Regularized Neural Networks](https://mlsanity.com/paper/2006.00294/) (2020)
- [Adversarial Risk Bounds via Function Transformation](https://mlsanity.com/paper/1810.09519/) (2018)
- [Semi-supervised Vector-valued Learning: Improved Bounds and Algorithms](https://mlsanity.com/paper/1909.04883/) (2019)

## Built on by (most cited first, 1 of 1)

- [Dynamics to decision: A mathematical theory of Lyapunov spectra and decision boundaries in deep classifiers](https://mlsanity.com/paper/2609.39190/) (2026)

## Abstract (arXiv)

Powerful deep neural networks are vulnerable to adversarial attacks. To obtain adversarially robust models, researchers have separately developed adversarial training and Jacobian regularization techniques. There are abundant theoretical and empirical studies for adversarial training, but theoretical foundations for Jacobian regularization are still lacking. In this study, we show that Jacobian regularization is closely related to adversarial training in that ℓ2 or ℓ1 Jacobian regularized loss serves as an approximate upper bound on the adversarially robust loss under ℓ2 or ℓ∞ adversarial attack respectively. Further, we establish the robust generalization gap for Jacobian regularized risk minimizer via bounding the Rademacher complexity of both the standard loss function class and Jacobian regularization function class. Our theoretical results indicate that the norms of Jacobian are related to both standard and robust generalization. We also perform experiments on MNIST data classification to demonstrate that Jacobian regularized risk minimization indeed serves as a surrogate for adversarially robust risk minimization, and that reducing the norms of Jacobian can improve both standard and robust generalization. This study promotes both theoretical and empirical understandings to adversarially robust generalization via Jacobian regularization.