# Amata: An Annealing Mechanism for Adversarial Training Acceleration

arXiv: https://arxiv.org/abs/2012.08112 · HTML: https://mlsanity.com/paper/2012.08112/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2020.

## Built on by (most cited first, 5 of 5)

- [SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustness](https://mlsanity.com/paper/2207.12391/) (2022)
- [Towards Robust Neural Networks via Close-loop Control](https://mlsanity.com/paper/2102.01862/) (2021)
- [Hard Adversarial Example Mining for Improving Robust Fairness](https://mlsanity.com/paper/2308.01823/) (2023)
- [Dynamic Efficient Adversarial Training Guided by Gradient Magnitude](https://mlsanity.com/paper/2103.03076/) (2021)
- [ScaLA: Accelerating Adaptation of Pre-Trained Transformer-Based Language Models via Efficient Large-Batch Adversarial Noise](https://mlsanity.com/paper/2201.12469/) (2022)

## Abstract (arXiv)

Despite the empirical success in various domains, it has been revealed that deep neural networks are vulnerable to maliciously perturbed input data that much degrade their performance. This is known as adversarial attacks. To counter adversarial attacks, adversarial training formulated as a form of robust optimization has been demonstrated to be effective. However, conducting adversarial training brings much computational overhead compared with standard training. In order to reduce the computational cost, we propose an annealing mechanism, Amata, to reduce the overhead associated with adversarial training. The proposed Amata is provably convergent, well-motivated from the lens of optimal control theory and can be combined with existing acceleration methods to further enhance performance. It is demonstrated that on standard datasets, Amata can achieve similar or better robustness with around 1/3 to 1/2 the computational time compared with traditional methods. In addition, Amata can be incorporated into other adversarial training acceleration algorithms (e.g. YOPO, Free, Fast, and ATTA), which leads to further reduction in computational time on large-scale problems.