# Adversarial Training with Stochastic Weight Average

arXiv: https://arxiv.org/abs/2009.10526 · HTML: https://mlsanity.com/paper/2009.10526/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2020.

## Built on by (most cited first, 12 of 12)

- [Relating Adversarially Robust Generalization to Flat Minima](https://mlsanity.com/paper/2104.04448/) (2021)
- [Improving Fast Adversarial Training with Prior-Guided Knowledge](https://mlsanity.com/paper/2304.00202/) (2023)
- [Stability Analysis and Generalization Bounds of Adversarial Training](https://mlsanity.com/paper/2210.00960/) (2022)
- [Stochastic Weight Averaging Revisited](https://mlsanity.com/paper/2201.00519/) (2022)
- [PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor](https://mlsanity.com/paper/2403.06668/) (2024)
- [SRoUDA: Meta Self-training for Robust Unsupervised Domain Adaptation](https://mlsanity.com/paper/2212.05917/) (2022)
- [Conserve-Update-Revise to Cure Generalization and Robustness Trade-off in Adversarial Training](https://mlsanity.com/paper/2401.14948/) (2024)
- [Generating Less Certain Adversarial Examples Improves Robust Generalization](https://mlsanity.com/paper/2310.04539/) (2023)
- [Criticality Leveraged Adversarial Training (CLAT) for Boosted Performance via Parameter Efficiency](https://mlsanity.com/paper/2408.10204/) (2024)
- [Towards Trustworthy Unsupervised Domain Adaptation: A Representation Learning Perspective for Enhancing Robustness, Discrimination, and Generalization](https://mlsanity.com/paper/2406.13180/) (2024)
- [Proxy Robustness in Vision Language Models is Effortlessly Transferable](https://mlsanity.com/paper/2601.12865/) (2026)
- [Bayesian Generative Adversarial Networks via Gaussian Approximation for Tabular Data Synthesis](https://mlsanity.com/paper/2602.21948/) (2026)

## Abstract (arXiv)

Adversarial training deep neural networks often experience serious overfitting problem. Recently, it is explained that the overfitting happens because the sample complexity of training data is insufficient to generalize robustness. In traditional machine learning, one way to relieve overfitting from the lack of data is to use ensemble methods. However, adversarial training multiple networks is extremely expensive. Moreover, we found that there is a dilemma on choosing target model to generate adversarial examples. Optimizing attack to the members of ensemble will be suboptimal attack to the ensemble and incurs covariate shift, while attack to ensemble will weaken the members and lose the benefit from ensembling. In this paper, we propose adversarial training with Stochastic weight average (SWA); while performing adversarial training, we aggregate the temporal weight states in the trajectory of training. By adopting SWA, the benefit of ensemble can be gained without tremendous computational increment and without facing the dilemma. Moreover, we further improved SWA to be adequate to adversarial training. The empirical results on CIFAR-10, CIFAR-100 and SVHN show that our method can improve the robustness of models.