# SAT: Improving Adversarial Training via Curriculum-Based Loss Smoothing

arXiv: https://arxiv.org/abs/2003.09347 · HTML: https://mlsanity.com/paper/2003.09347/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2020.

## Built on by (most cited first, 26 of 26)

- [LAS-AT: Adversarial Training with Learnable Attack Strategy](https://mlsanity.com/paper/2203.06616/) (2022)
- [How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses](https://mlsanity.com/paper/2305.10862/) (2023)
- [Constrained Learning with Non-Convex Losses](https://mlsanity.com/paper/2103.05134/) (2021)
- [Navigating Complexity: Toward Lossless Graph Condensation via Expanding Window Matching](https://mlsanity.com/paper/2402.05011/) (2024)
- [Robust Models are less Over-Confident](https://mlsanity.com/paper/2210.05938/) (2022)
- [CNN Filter DB: An Empirical Investigation of Trained Convolutional Filters](https://mlsanity.com/paper/2203.15331/) (2022)
- [Robust Deep Reinforcement Learning through Bootstrapped Opportunistic Curriculum](https://mlsanity.com/paper/2206.10057/) (2022)
- [Revisiting and Exploring Efficient Fast Adversarial Training via LAW: Lipschitz Regularization and Auto Weight Averaging](https://mlsanity.com/paper/2308.11443/) (2023)
- [Diversified Adversarial Attacks based on Conjugate Gradient Method](https://mlsanity.com/paper/2206.09628/) (2022)
- [Enhancing Adversarial Robustness for Deep Metric Learning](https://mlsanity.com/paper/2203.01439/) (2022)
- [Adversarial Robustness through the Lens of Convolutional Filters](https://mlsanity.com/paper/2204.02481/) (2022)
- [Adversarial Finetuning with Latent Representation Constraint to Mitigate Accuracy-Robustness Tradeoff](https://mlsanity.com/paper/2308.16454/) (2023)
- [Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off](https://mlsanity.com/paper/2402.14648/) (2024)
- [Conserve-Update-Revise to Cure Generalization and Robustness Trade-off in Adversarial Training](https://mlsanity.com/paper/2401.14948/) (2024)
- [Splitting the Difference on Adversarial Training](https://mlsanity.com/paper/2310.02480/) (2023)
- [Catastrophic Overfitting: A Potential Blessing in Disguise](https://mlsanity.com/paper/2402.18211/) (2024)
- [Dynamic Label Adversarial Training for Deep Learning Robustness Against Adversarial Attacks](https://mlsanity.com/paper/2408.13102/) (2024)
- [Robustness to Multi-Modal Environment Uncertainty in MARL using Curriculum Learning](https://mlsanity.com/paper/2310.08746/) (2023)
- [Topology-preserving Adversarial Training for Alleviating Natural Accuracy Degradation](https://mlsanity.com/paper/2311.17607/) (2023)
- [D2R: dual regularization loss with collaborative adversarial generation for model robustness](https://mlsanity.com/paper/2506.07056/) (2025)
- [AdaGAT: Adaptive Guidance Adversarial Training for the Robustness of Deep Neural Networks](https://mlsanity.com/paper/2508.17265/) (2025)
- [Advancing Adversarial Robustness Through Adversarial Logit Update](https://mlsanity.com/paper/2308.15072/) (2023)
- [Harmonizing Feature Maps: A Graph Convolutional Approach for Enhancing Adversarial Robustness](https://mlsanity.com/paper/2406.11576/) (2024)
- [Sustainable Self-evolution Adversarial Training](https://mlsanity.com/paper/2412.02270/) (2024)
- [How Does the Smoothness Approximation Method Facilitate Generalization for Federated Adversarial Learning?](https://mlsanity.com/paper/2412.08282/) (2024)
- [On the Escaping Efficiency of Distributed Adversarial Training Algorithms](https://mlsanity.com/paper/2509.11337/) (2025)

## Abstract (arXiv)

Adversarial training (AT) has become a popular choice for training robust networks. However, it tends to sacrifice clean accuracy heavily in favor of robustness and suffers from a large generalization error. To address these concerns, we propose Smooth Adversarial Training (SAT), guided by our analysis on the eigenspectrum of the loss Hessian. We find that curriculum learning, a scheme that emphasizes on starting "easy" and gradually ramping up on the "difficulty" of training, smooths the adversarial loss landscape for a suitably chosen difficulty metric. We present a general formulation for curriculum learning in the adversarial setting and propose two difficulty metrics based on the maximal Hessian eigenvalue (H-SAT) and the softmax probability (P-SA). We demonstrate that SAT stabilizes network training even for a large perturbation norm and allows the network to operate at a better clean accuracy versus robustness trade-off curve compared to AT. This leads to a significant improvement in both clean accuracy and robustness compared to AT, TRADES, and other baselines. To highlight a few results, our best model improves normal and robust accuracy by 6% and 1% on CIFAR-100 compared to AT, respectively. On Imagenette, a ten-class subset of ImageNet, our model outperforms AT by 23% and 3% on normal and robust accuracy respectively.