# Robustness via curvature regularization, and vice versa

arXiv: https://arxiv.org/abs/1811.09716 · HTML: https://mlsanity.com/paper/1811.09716/

## Key facts

- Cited by 175 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2018.

## Built on by (most cited first, 30 of 175)

- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Efficient Sharpness-aware Minimization for Improved Training of Neural Networks](https://mlsanity.com/paper/2110.03141/) (2021)
- [Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2103.08896/) (2021)
- [Sharpness-Aware Training for Free](https://mlsanity.com/paper/2205.14083/) (2022)
- [Learning Smooth Neural Functions via Lipschitz Regularization](https://mlsanity.com/paper/2202.08345/) (2022)
- [Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond](https://mlsanity.com/paper/2502.05374/) (2025)
- [When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?](https://mlsanity.com/paper/2111.01124/) (2021)
- [Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization](https://mlsanity.com/paper/2112.12376/) (2021)
- [On the effectiveness of adversarial training against common corruptions](https://mlsanity.com/paper/2103.02325/) (2021)
- [Global Vision Transformer Pruning with Hessian-Aware Saliency](https://mlsanity.com/paper/2110.04869/) (2021)
- [How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses](https://mlsanity.com/paper/2305.10862/) (2023)
- [A Dynamical System Perspective for Lipschitz Neural Networks](https://mlsanity.com/paper/2110.12690/) (2021)
- [Probabilistic Margins for Instance Reweighting in Adversarial Training](https://mlsanity.com/paper/2106.07904/) (2021)
- [Exploring Memorization in Adversarial Training](https://mlsanity.com/paper/2106.01606/) (2021)
- [Subspace Adversarial Training](https://mlsanity.com/paper/2111.12229/) (2021)
- [Sparsity Winning Twice: Better Robust Generalization from More Efficient Training](https://mlsanity.com/paper/2202.09844/) (2022)
- [Pyramid Adversarial Training Improves ViT Performance](https://mlsanity.com/paper/2111.15121/) (2021)
- [Training with More Confidence: Mitigating Injected and Natural Backdoors During Training](https://mlsanity.com/paper/2202.06382/) (2022)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [TANGOS: Regularizing Tabular Neural Networks through Gradient Orthogonalization and Specialization](https://mlsanity.com/paper/2303.05506/) (2023)
- [Low Curvature Activations Reduce Overfitting in Adversarial Training](https://mlsanity.com/paper/2102.07861/) (2021)
- [Reliably fast adversarial training via latent adversarial perturbation](https://mlsanity.com/paper/2104.01575/) (2021)
- [Defending Against Image Corruptions Through Adversarial Augmentations](https://mlsanity.com/paper/2104.01086/) (2021)
- [AdvRush: Searching for Adversarially Robust Neural Architectures](https://mlsanity.com/paper/2108.01289/) (2021)
- [SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretability](https://mlsanity.com/paper/2208.09418/) (2022)
- [Randomized Adversarial Training via Taylor Expansion](https://mlsanity.com/paper/2303.10653/) (2023)
- [Noisy Feature Mixup](https://mlsanity.com/paper/2110.02180/) (2021)
- [A Unified Analysis of Mixed Sample Data Augmentation: A Loss Function Perspective](https://mlsanity.com/paper/2208.09913/) (2022)
- [Memorization Through the Lens of Curvature of Loss Function Around Samples](https://mlsanity.com/paper/2307.05831/) (2023)

## Abstract (arXiv)

State-of-the-art classifiers have been shown to be largely vulnerable to adversarial perturbations. One of the most effective strategies to improve robustness is adversarial training. In this paper, we investigate the effect of adversarial training on the geometry of the classification landscape and decision boundaries. We show in particular that adversarial training leads to a significant decrease in the curvature of the loss surface with respect to inputs, leading to a drastically more "linear" behaviour of the network. Using a locally quadratic approximation, we provide theoretical evidence on the existence of a strong relation between large robustness and small curvature. To further show the importance of reduced curvature for improving the robustness, we propose a new regularizer that directly minimizes curvature of the loss surface, and leads to adversarial robustness that is on par with adversarial training. Besides being a more efficient and principled alternative to adversarial training, the proposed regularizer confirms our claims on the importance of exhibiting quasi-linear behavior in the vicinity of data points in order to achieve robustness.