# Scaling Compute Is Not All You Need for Adversarial Robustness

arXiv: https://arxiv.org/abs/2312.13131 · HTML: https://mlsanity.com/paper/2312.13131/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2023.

## Built on by (most cited first, 6 of 6)

- [Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies](https://mlsanity.com/paper/2404.09349/) (2024)
- [Towards Efficient Neuro-Symbolic AI: From Workload Characterization to Hardware Architecture](https://mlsanity.com/paper/2409.13153/) (2024)
- [The Price of Implicit Bias in Adversarially Robust Generalization](https://mlsanity.com/paper/2406.04981/) (2024)
- [Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems](https://mlsanity.com/paper/2410.23472/) (2024)
- [Complexity Matters: Effective Dimensionality as a Measure for Adversarial Robustness](https://mlsanity.com/paper/2410.18556/) (2024)
- [Disentangling Safe and Unsafe Corruptions via Anisotropy and Locality](https://mlsanity.com/paper/2501.18098/) (2025)

## Abstract (arXiv)

The last six years have witnessed significant progress in adversarially robust deep learning. As evidenced by the CIFAR-10 dataset category in RobustBench benchmark, the accuracy under ℓ_∞ adversarial perturbations improved from 44% in citetMadry2018Towards to 71% in citetpeng2023robust. Although impressive, existing state-of-the-art is still far from satisfactory. It is further observed that best-performing models are often very large models adversarially trained by industrial labs with significant computational budgets. In this paper, we aim to understand: ``how much longer can computing power drive adversarial robustness advances?" To answer this question, we derive scaling laws for adversarial robustness which can be extrapolated in the future to provide an estimate of how much cost we would need to pay to reach a desired level of robustness. We show that increasing the FLOPs needed for adversarial training does not bring as much advantage as it does for standard training in terms of performance improvements. Moreover, we find that some of the top-performing techniques are difficult to exactly reproduce, suggesting that they are not robust enough for minor changes in the training setup. Our analysis also uncovers potentially worthwhile directions to pursue in future research. Finally, we make our benchmarking framework (built on top of texttttimm~citeprw2019timm) publicly available to facilitate future analysis in efficient robust deep learning.