# Adversarial Distributional Training for Robust Deep Learning

arXiv: https://arxiv.org/abs/2002.05999 · HTML: https://mlsanity.com/paper/2002.05999/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2020.

## Built on by (most cited first, 30 of 79)

- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Better Diffusion Models Further Improve Adversarial Training](https://mlsanity.com/paper/2302.04638/) (2023)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks](https://mlsanity.com/paper/2110.03825/) (2021)
- [Black-box Detection of Backdoor Attacks with Limited Information and Data](https://mlsanity.com/paper/2103.13127/) (2021)
- [Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training](https://mlsanity.com/paper/2102.04716/) (2021)
- [Exploring Memorization in Adversarial Training](https://mlsanity.com/paper/2106.01606/) (2021)
- [ViewFool: Evaluating the Robustness of Visual Recognition to Adversarial Viewpoints](https://mlsanity.com/paper/2210.03895/) (2022)
- [Robust Principles: Architectural Design Principles for Adversarially Robust CNNs](https://mlsanity.com/paper/2308.16258/) (2023)
- [Enhancing Adversarial Training with Second-Order Statistics of Weights](https://mlsanity.com/paper/2203.06020/) (2022)
- [A Unified Wasserstein Distributional Robustness Framework for Adversarial Training](https://mlsanity.com/paper/2202.13437/) (2022)
- [Randomized Adversarial Training via Taylor Expansion](https://mlsanity.com/paper/2303.10653/) (2023)
- [Isometric 3D Adversarial Examples in the Physical World](https://mlsanity.com/paper/2210.15291/) (2022)
- [Adaptive Image Transformations for Transfer-based Adversarial Attack](https://mlsanity.com/paper/2111.13844/) (2021)
- [Towards Viewpoint-Invariant Visual Recognition via Adversarial Training](https://mlsanity.com/paper/2307.10235/) (2023)
- [Recognizing Object by Components with Human Prior Knowledge Enhances Adversarial Robustness of Deep Neural Networks](https://mlsanity.com/paper/2212.01806/) (2022)
- [Feature Separation and Recalibration for Adversarial Robustness](https://mlsanity.com/paper/2303.13846/) (2023)
- [Wasserstein distributional robustness of neural networks](https://mlsanity.com/paper/2306.09844/) (2023)
- [Strong Transferable Adversarial Attacks via Ensembled Asymptotically Normal Distribution Learning](https://mlsanity.com/paper/2209.11964/) (2022)
- [Defenses in Adversarial Machine Learning: A Survey](https://mlsanity.com/paper/2312.08890/) (2023)
- [Fundamental Tradeoffs in Distributionally Adversarial Training](https://mlsanity.com/paper/2101.06309/) (2021)
- [Learning Diverse-Structured Networks for Adversarial Robustness](https://mlsanity.com/paper/2102.01886/) (2021)
- [Global-Local Regularization Via Distributional Robustness](https://mlsanity.com/paper/2203.00553/) (2022)
- [Outlier Robust Adversarial Training](https://mlsanity.com/paper/2309.05145/) (2023)
- [Rethinking Natural Adversarial Examples for Classification Models](https://mlsanity.com/paper/2102.11731/) (2021)
- [Defending Black-box Skeleton-based Human Activity Classifiers](https://mlsanity.com/paper/2203.04713/) (2022)
- [Towards Robust Dataset Learning](https://mlsanity.com/paper/2211.10752/) (2022)
- [Robust Spatiotemporal Traffic Forecasting with Reinforced Dynamic Adversarial Training](https://mlsanity.com/paper/2306.14126/) (2023)
- [Focus on Hiders: Exploring Hidden Threats for Enhancing Adversarial Training](https://mlsanity.com/paper/2312.07067/) (2023)
- [On the regularized risk of distributionally robust learning over deep neural networks](https://mlsanity.com/paper/2109.06294/) (2021)

## Abstract (arXiv)

Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples. However, most existing AT methods adopt a specific attack to craft adversarial examples, leading to the unreliable robustness against other unseen attacks. Besides, a single attack algorithm could be insufficient to explore the space of perturbations. In this paper, we introduce adversarial distributional training (ADT), a novel framework for learning robust models. ADT is formulated as a minimax optimization problem, where the inner maximization aims to learn an adversarial distribution to characterize the potential adversarial examples around a natural one under an entropic regularizer, and the outer minimization aims to train robust models by minimizing the expected loss over the worst-case adversarial distributions. Through a theoretical analysis, we develop a general algorithm for solving ADT, and present three approaches for parameterizing the adversarial distributions, ranging from the typical Gaussian distributions to the flexible implicit ones. Empirical results on several benchmarks validate the effectiveness of ADT compared with the state-of-the-art AT methods.