A Flat Minima Perspective on Understanding Augmentations and Model Robustness
Not on the map. Why?
Abstract (arXiv)
Model robustness indicates a model's capability to generalize well on unforeseen distributional shifts, including data corruptions and adversarial attacks. Data augmentation is one of the most prevalent and effective ways to enhance robustness. Despite the great success of the diverse augmentations in different fields, a unified theoretical understanding of their efficacy in improving model robustness is lacking. We theoretically reveal a general condition for label-preserving augmentations to bring robustness to diverse distribution shifts through the lens of flat minima and generalization bound, which de facto turns out to be strongly correlated with robustness against different distribution shifts in practice. Unlike most earlier works, our theoretical framework accommodates all the label-preserving augmentations and is not limited to particular distribution shifts. We substantiate our theories through different simulations on the existing common corruption and adversarial robustness benchmarks based on the CIFAR and ImageNet datasets.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 60 of its 79 references that are in the map. First posted on arXiv 30 May 2025.
Builds on
60 of 79 references.
References · 60
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
- 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
- 2014Conditional Generative Adversarial Netscited by 2,162
- 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
- 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
38 more
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
- 2020In Search of Lost Domain Generalizationcited by 1,054
- 2014Striving for Simplicity: The All Convolutional Netcited by 990
- 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
- 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
- 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
- 2018On Lazy Training in Differentiable Programmingcited by 675
- 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
- 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
- 2019Fantastic Generalization Measures and Where to Find Themcited by 513
- 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
- 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 434
- 2019A Fourier Perspective on Model Robustness in Computer Visioncited by 288
- 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 261
- 2019Fast AutoAugmentcited by 243
- 2020Deep Domain-Adversarial Image Generation for Domain Generalisationcited by 231
- 2017Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-supervised Object and Action Localizationcited by 207
- 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 204
- 2021Data Augmentation Can Improve Robustnesscited by 194
- 2019Are Labels Required for Improving Adversarial Robustness?cited by 163
- 2021PixMix: Dreamlike Pictures Comprehensively Improve Safety Measurescited by 131
- 2020Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustnesscited by 121
- 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
- 2019Adversarial AutoAugmentcited by 111
- 2021On Interaction Between Augmentations and Corruptions in Natural Corruption Robustnesscited by 97
- 2022A Comprehensive Survey of Image Augmentation Techniques for Deep Learningcited by 97
- 2019Faster AutoAugment: Learning Augmentation Strategies using Backpropagationcited by 96
- 2018BAGAN: Data Augmentation with Balancing GANcited by 90
- 2021AugMax: Adversarial Composition of Random Augmentations for Robust Trainingcited by 90
- 2018Style Augmentation: Data Augmentation via Style Randomizationcited by 79
- 2022When Do Flat Minima Optimizers Work?cited by 78
- 2024DiffuseMix: Label-Preserving Data Augmentation with Diffusion Modelscited by 78
- 2019Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networkscited by 76
- 2016A Boundary Tilting Persepective on the Phenomenon of Adversarial Examplescited by 70
- 2019Robustness to Adversarial Perturbations in Learning from Incomplete Datacited by 49
- 2022Rank Diminishing in Deep Neural Networkscited by 47
- 2023Data Augmentation Alone Can Improve Adversarial Trainingcited by 43
50 most cited of 60 All 60 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.