# Efficient Robust Training via Backward Smoothing

arXiv: https://arxiv.org/abs/2010.01278 · HTML: https://mlsanity.com/paper/2010.01278/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2020.

## Built on by (most cited first, 29 of 29)

- [Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization](https://mlsanity.com/paper/2112.12376/) (2021)
- [How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses](https://mlsanity.com/paper/2305.10862/) (2023)
- [An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learning](https://mlsanity.com/paper/2308.00788/) (2023)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [Exploring Misclassifications of Robust Neural Networks to Enhance Adversarial Attacks](https://mlsanity.com/paper/2105.10304/) (2021)
- [Attacking Adversarial Attacks as A Defense](https://mlsanity.com/paper/2106.04938/) (2021)
- [One-shot Neural Backdoor Erasing via Adversarial Weight Masking](https://mlsanity.com/paper/2207.04497/) (2022)
- [Enhancing Adversarial Robustness via Test-time Transformation Ensembling](https://mlsanity.com/paper/2107.14110/) (2021)
- [Robust Models are less Over-Confident](https://mlsanity.com/paper/2210.05938/) (2022)
- [CNN Filter DB: An Empirical Investigation of Trained Convolutional Filters](https://mlsanity.com/paper/2203.15331/) (2022)
- [Uncovering Adversarial Risks of Test-Time Adaptation](https://mlsanity.com/paper/2301.12576/) (2023)
- [On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training](https://mlsanity.com/paper/2112.07324/) (2021)
- [RoCourseNet: Distributionally Robust Training of a Prediction Aware Recourse Model](https://mlsanity.com/paper/2206.00700/) (2022)
- [Diversified Adversarial Attacks based on Conjugate Gradient Method](https://mlsanity.com/paper/2206.09628/) (2022)
- [Adversarial Robustness through the Lens of Convolutional Filters](https://mlsanity.com/paper/2204.02481/) (2022)
- [BulletTrain: Accelerating Robust Neural Network Training via Boundary Example Mining](https://mlsanity.com/paper/2109.14707/) (2021)
- [Machine Learning Robustness: A Primer](https://mlsanity.com/paper/2404.00897/) (2024)
- [Adversarial Finetuning with Latent Representation Constraint to Mitigate Accuracy-Robustness Tradeoff](https://mlsanity.com/paper/2308.16454/) (2023)
- [Conserve-Update-Revise to Cure Generalization and Robustness Trade-off in Adversarial Training](https://mlsanity.com/paper/2401.14948/) (2024)
- [Robust Binary Models by Pruning Randomly-initialized Networks](https://mlsanity.com/paper/2202.01341/) (2022)
- [Revisiting Outer Optimization in Adversarial Training](https://mlsanity.com/paper/2209.01199/) (2022)
- [Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-fitting Perspective](https://mlsanity.com/paper/2302.11963/) (2023)
- [Exploring Robust Features for Improving Adversarial Robustness](https://mlsanity.com/paper/2309.04650/) (2023)
- [Bridged Adversarial Training](https://mlsanity.com/paper/2108.11135/) (2021)
- [Bag of Tricks for FGSM Adversarial Training](https://mlsanity.com/paper/2209.02684/) (2022)
- [IRAD: Implicit Representation-driven Image Resampling against Adversarial Attacks](https://mlsanity.com/paper/2310.11890/) (2023)
- [Democratic Training Against Universal Adversarial Perturbations](https://mlsanity.com/paper/2502.05542/) (2025)
- [Generating Adversarial Samples in Mini-Batches May Be Detrimental To Adversarial Robustness](https://mlsanity.com/paper/2303.17720/) (2023)
- [Efficient Semi-Supervised Adversarial Training via Latent Clustering-Based Data Reduction](https://mlsanity.com/paper/2501.10466/) (2025)

## Abstract (arXiv)

Adversarial training is so far the most effective strategy in defending against adversarial examples. However, it suffers from high computational costs due to the iterative adversarial attacks in each training step. Recent studies show that it is possible to achieve fast Adversarial Training by performing a single-step attack with random initialization. However, such an approach still lags behind state-of-the-art adversarial training algorithms on both stability and model robustness. In this work, we develop a new understanding towards Fast Adversarial Training, by viewing random initialization as performing randomized smoothing for better optimization of the inner maximization problem. Following this new perspective, we also propose a new initialization strategy, backward smoothing, to further improve the stability and model robustness over single-step robust training methods. Experiments on multiple benchmarks demonstrate that our method achieves similar model robustness as the original TRADES method while using much less training time (~3x improvement with the same training schedule).