# MMA Training: Direct Input Space Margin Maximization through Adversarial Training

arXiv: https://arxiv.org/abs/1812.02637 · HTML: https://mlsanity.com/paper/1812.02637/

## Key facts

- Cited by 172 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2018.

## Built on by (most cited first, 30 of 172)

- [Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2205.07460/) (2022)
- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Cross-Entropy Loss Functions: Theoretical Analysis and Applications](https://mlsanity.com/paper/2304.07288/) (2023)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Robustness and Accuracy Could Be Reconcilable by (Proper) Definition](https://mlsanity.com/paper/2202.10103/) (2022)
- [Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks](https://mlsanity.com/paper/2110.03825/) (2021)
- [Relating Adversarially Robust Generalization to Flat Minima](https://mlsanity.com/paper/2104.04448/) (2021)
- [Reliable Adversarial Distillation with Unreliable Teachers](https://mlsanity.com/paper/2106.04928/) (2021)
- [CFA: Class-wise Calibrated Fair Adversarial Training](https://mlsanity.com/paper/2303.14460/) (2023)
- [Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspective](https://mlsanity.com/paper/2210.01787/) (2022)
- [Adversarial Robustness under Long-Tailed Distribution](https://mlsanity.com/paper/2104.02703/) (2021)
- [Robust Principles: Architectural Design Principles for Adversarially Robust CNNs](https://mlsanity.com/paper/2308.16258/) (2023)
- [Practical Evaluation of Adversarial Robustness via Adaptive Auto Attack](https://mlsanity.com/paper/2203.05154/) (2022)
- [Enhancing Adversarial Training with Second-Order Statistics of Weights](https://mlsanity.com/paper/2203.06020/) (2022)
- [Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neurons](https://mlsanity.com/paper/2102.05363/) (2021)
- [Adversarial Robustness with Semi-Infinite Constrained Learning](https://mlsanity.com/paper/2110.15767/) (2021)
- [Self-Ensemble Adversarial Training for Improved Robustness](https://mlsanity.com/paper/2203.09678/) (2022)
- [Exploring and Exploiting Decision Boundary Dynamics for Adversarial Robustness](https://mlsanity.com/paper/2302.03015/) (2023)
- [Generalized Strategic Classification and the Case of Aligned Incentives](https://mlsanity.com/paper/2202.04357/) (2022)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [Exploring Misclassifications of Robust Neural Networks to Enhance Adversarial Attacks](https://mlsanity.com/paper/2105.10304/) (2021)
- [Attacking Adversarial Attacks as A Defense](https://mlsanity.com/paper/2106.04938/) (2021)
- [Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives](https://mlsanity.com/paper/2307.16851/) (2023)
- [A Review of Adversarial Attack and Defense for Classification Methods](https://mlsanity.com/paper/2111.09961/) (2021)
- [Randomized Adversarial Training via Taylor Expansion](https://mlsanity.com/paper/2303.10653/) (2023)
- [Taxonomy of Machine Learning Safety: A Survey and Primer](https://mlsanity.com/paper/2106.04823/) (2021)
- [Fast Adversarial Training with Adaptive Step Size](https://mlsanity.com/paper/2206.02417/) (2022)
- [Robust Models are less Over-Confident](https://mlsanity.com/paper/2210.05938/) (2022)
- [LAFEAT: Piercing Through Adversarial Defenses with Latent Features](https://mlsanity.com/paper/2104.09284/) (2021)
- [Boosting the Certified Robustness of L-infinity Distance Nets](https://mlsanity.com/paper/2110.06850/) (2021)

## Abstract (arXiv)

We study adversarial robustness of neural networks from a margin maximization perspective, where margins are defined as the distances from inputs to a classifier's decision boundary. Our study shows that maximizing margins can be achieved by minimizing the adversarial loss on the decision boundary at the "shortest successful perturbation", demonstrating a close connection between adversarial losses and the margins. We propose Max-Margin Adversarial (MMA) training to directly maximize the margins to achieve adversarial robustness. Instead of adversarial training with a fixed ε, MMA offers an improvement by enabling adaptive selection of the "correct" ε as the margin individually for each datapoint. In addition, we rigorously analyze adversarial training with the perspective of margin maximization, and provide an alternative interpretation for adversarial training, maximizing either a lower or an upper bound of the margins. Our experiments empirically confirm our theory and demonstrate MMA training's efficacy on the MNIST and CIFAR10 datasets w.r.t. ℓ_∞ and ℓ2 robustness. Code and models are available at https://github.com/BorealisAI/mmatraining.