# Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness

arXiv: https://arxiv.org/abs/2010.08001 · HTML: https://mlsanity.com/paper/2010.08001/

## Key facts

- Cited by 121 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2020.

## Built on by (most cited first, 30 of 121)

- [SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2310.03684/) (2023)
- [Improving Out-of-Distribution Robustness via Selective Augmentation](https://mlsanity.com/paper/2201.00299/) (2022)
- [Learning to Diversify for Single Domain Generalization](https://mlsanity.com/paper/2108.11726/) (2021)
- [Progressive Domain Expansion Network for Single Domain Generalization](https://mlsanity.com/paper/2103.16050/) (2021)
- [AugMax: Adversarial Composition of Random Augmentations for Robust Training](https://mlsanity.com/paper/2110.13771/) (2021)
- [Adversarially Adaptive Normalization for Single Domain Generalization](https://mlsanity.com/paper/2106.01899/) (2021)
- [Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation](https://mlsanity.com/paper/2207.04892/) (2022)
- [DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations](https://mlsanity.com/paper/2201.09637/) (2022)
- [Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization](https://mlsanity.com/paper/2106.06916/) (2021)
- [Training on Thin Air: Improve Image Classification with Generated Data](https://mlsanity.com/paper/2305.15316/) (2023)
- [A Systematic Review of Robustness in Deep Learning for Computer Vision: Mind the gap?](https://mlsanity.com/paper/2112.00639/) (2021)
- [Meta-causal Learning for Single Domain Generalization](https://mlsanity.com/paper/2304.03709/) (2023)
- [Uncertainty-guided Model Generalization to Unseen Domains](https://mlsanity.com/paper/2103.07531/) (2021)
- [Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models](https://mlsanity.com/paper/2310.03182/) (2023)
- [Attention Consistency on Visual Corruptions for Single-Source Domain Generalization](https://mlsanity.com/paper/2204.13091/) (2022)
- [StyLIP: Multi-Scale Style-Conditioned Prompt Learning for CLIP-based Domain Generalization](https://mlsanity.com/paper/2302.09251/) (2023)
- [Progressive Random Convolutions for Single Domain Generalization](https://mlsanity.com/paper/2304.00424/) (2023)
- [Improving Single Domain-Generalized Object Detection: A Focus on Diversification and Alignment](https://mlsanity.com/paper/2405.14497/) (2024)
- [Ranking Distance Calibration for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2112.00260/) (2021)
- [Taxonomy of Machine Learning Safety: A Survey and Primer](https://mlsanity.com/paper/2106.04823/) (2021)
- [Prompt-Driven Dynamic Object-Centric Learning for Single Domain Generalization](https://mlsanity.com/paper/2402.18447/) (2024)
- [Style-Hallucinated Dual Consistency Learning: A Unified Framework for Visual Domain Generalization](https://mlsanity.com/paper/2212.09068/) (2022)
- [A Survey on the Robustness of Computer Vision Models against Common Corruptions](https://mlsanity.com/paper/2305.06024/) (2023)
- [Domain Generalization via Balancing Training Difficulty and Model Capability](https://mlsanity.com/paper/2309.00844/) (2023)
- [Distribution-Aware Data Expansion with Diffusion Models](https://mlsanity.com/paper/2403.06741/) (2024)
- [Out-of-Domain Generalization from a Single Source: An Uncertainty Quantification Approach](https://mlsanity.com/paper/2108.02888/) (2021)
- [Robust Domain Generalization for Multi-modal Object Recognition](https://mlsanity.com/paper/2408.05831/) (2024)
- [Not Just Pretty Pictures: Toward Interventional Data Augmentation Using Text-to-Image Generators](https://mlsanity.com/paper/2212.11237/) (2022)
- [Bootstrap Generalization Ability from Loss Landscape Perspective](https://mlsanity.com/paper/2209.08473/) (2022)
- [Adversarial AutoMixup](https://mlsanity.com/paper/2312.11954/) (2023)

## Abstract (arXiv)

Adversarial data augmentation has shown promise for training robust deep neural networks against unforeseen data shifts or corruptions. However, it is difficult to define heuristics to generate effective fictitious target distributions containing "hard" adversarial perturbations that are largely different from the source distribution. In this paper, we propose a novel and effective regularization term for adversarial data augmentation. We theoretically derive it from the information bottleneck principle, which results in a maximum-entropy formulation. Intuitively, this regularization term encourages perturbing the underlying source distribution to enlarge predictive uncertainty of the current model, so that the generated "hard" adversarial perturbations can improve the model robustness during training. Experimental results on three standard benchmarks demonstrate that our method consistently outperforms the existing state of the art by a statistically significant margin.