# Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies

arXiv: https://arxiv.org/abs/2404.09349 · HTML: https://mlsanity.com/paper/2404.09349/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 50 references that are in the map.
- First posted on arXiv 14 Apr 2024.

## Builds on (most cited first, 30 of 41)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [Beyond neural scaling laws: beating power law scaling via data pruning](https://mlsanity.com/paper/2206.14486/) (2022)
- [Scaling Data-Constrained Language Models](https://mlsanity.com/paper/2305.16264/) (2023)
- [Adversarially Robust Generalization Requires More Data](https://mlsanity.com/paper/1804.11285/) (2018)
- [TrustLLM: Trustworthiness in Large Language Models](https://mlsanity.com/paper/2401.05561/) (2024)
- [Improving Robustness using Generated Data](https://mlsanity.com/paper/2110.09468/) (2021)
- [Better Diffusion Models Further Improve Adversarial Training](https://mlsanity.com/paper/2302.04638/) (2023)
- [Data Augmentation Can Improve Robustness](https://mlsanity.com/paper/2111.05328/) (2021)
- [Image Hijacks: Adversarial Images can Control Generative Models at Runtime](https://mlsanity.com/paper/2309.00236/) (2023)
- [Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?](https://mlsanity.com/paper/2104.09425/) (2021)
- [Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models](https://mlsanity.com/paper/2211.17091/) (2022)
- [Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models](https://mlsanity.com/paper/2303.01870/) (2023)

## Built on by (most cited first, 30 of 38)

- [Artificial Kuramoto Oscillatory Neurons](https://mlsanity.com/paper/2410.13821/) (2024)
- [MeanSparse: Post-Training Robustness Enhancement Through Mean-Centered Feature Sparsification](https://mlsanity.com/paper/2406.05927/) (2024)
- [Double Visual Defense: Adversarial Pre-training and Instruction Tuning for Improving Vision-Language Model Robustness](https://mlsanity.com/paper/2501.09446/) (2025)
- [Understanding In-Context Learning of Linear Models in Transformers Through an Adversarial Lens](https://mlsanity.com/paper/2411.05189/) (2024)
- [Scaling Laws for Black box Adversarial Attacks](https://mlsanity.com/paper/2411.16782/) (2024)
- [Adversarial Alignment for LLMs Requires Simpler, Reproducible, and More Measurable Objectives](https://mlsanity.com/paper/2502.11910/) (2025)
- [Adversarial Attacks Leverage Interference Between Features in Superposition](https://mlsanity.com/paper/2510.11709/) (2025)
- [Intriguing Properties of Robust Classification](https://mlsanity.com/paper/2412.04245/) (2024)
- [GreedyPixel: Fine-Grained Black-Box Adversarial Attack Via Greedy Algorithm](https://mlsanity.com/paper/2501.14230/) (2025)
- [One Stone, Two Birds: Enhancing Adversarial Defense Through the Lens of Distributional Discrepancy](https://mlsanity.com/paper/2503.02169/) (2025)
- [Monitoring Robustness and Individual Fairness](https://mlsanity.com/paper/2506.00496/) (2025)
- [Robustness in Both Domains: CLIP Needs a Robust Text Encoder](https://mlsanity.com/paper/2506.03355/) (2025)
- [A Red Teaming Roadmap Towards System-Level Safety](https://mlsanity.com/paper/2506.05376/) (2025)
- [The Alignment Trap: Complexity Barriers](https://mlsanity.com/paper/2506.10304/) (2025)
- [DASH: A Meta-Attack Framework for Synthesizing Effective and Stealthy Adversarial Examples](https://mlsanity.com/paper/2508.13309/) (2025)
- [Complexity Matters: Effective Dimensionality as a Measure for Adversarial Robustness](https://mlsanity.com/paper/2410.18556/) (2024)
- [Adversarial Surrogate Risk Bounds for Binary Classification](https://mlsanity.com/paper/2506.09348/) (2025)
- [The Impact of Scaling Training Data on Adversarial Robustness](https://mlsanity.com/paper/2509.25927/) (2025)
- [Diffusion Models are Robust Pretrainers](https://mlsanity.com/paper/2511.02793/) (2025)
- [Solving adversarial examples requires solving exponential misalignment](https://mlsanity.com/paper/2603.03507/) (2026)
- [FastAT Benchmark: A Comprehensive Framework for Fair Evaluation of Fast Adversarial Training Methods](https://mlsanity.com/paper/2604.22853/) (2026)
- [Disentangling Safe and Unsafe Corruptions via Anisotropy and Locality](https://mlsanity.com/paper/2501.18098/) (2025)
- [Does simple trump complex? Comparing strategies for adversarial robustness in DNNs](https://mlsanity.com/paper/2508.18019/) (2025)
- [On the Escaping Efficiency of Distributed Adversarial Training Algorithms](https://mlsanity.com/paper/2509.11337/) (2025)
- [DeepProv: Behavioral Characterization and Repair of Neural Networks via Inference Provenance Graph Analysis](https://mlsanity.com/paper/2509.26562/) (2025)
- [Get RICH or Die Scaling: Profitably Trading Inference Compute for Robustness](https://mlsanity.com/paper/2510.06790/) (2025)
- [Tight Robustness Certificates and Wasserstein Distributional Attacks for Deep Neural Networks](https://mlsanity.com/paper/2510.10000/) (2025)
- [Sample-wise Adaptive Weighting for Transfer Consistency in Adversarial Distillation](https://mlsanity.com/paper/2512.10275/) (2025)
- [Expanding the Role of Diffusion Models for Robust Classifier Training](https://mlsanity.com/paper/2602.19931/) (2026)
- [Detecting Adversarial Data via Provable Adversarial Noise Amplification](https://mlsanity.com/paper/2605.02109/) (2026)

## Abstract (arXiv)

This paper revisits the simple, long-studied, yet still unsolved problem of making image classifiers robust to imperceptible perturbations. Taking CIFAR10 as an example, SOTA clean accuracy is about 100%, but SOTA robustness to ℓ∞-norm bounded perturbations barely exceeds 70%. To understand this gap, we analyze how model size, dataset size, and synthetic data quality affect robustness by developing the first scaling laws for adversarial training. Our scaling laws reveal inefficiencies in prior art and provide actionable feedback to advance the field. For instance, we discovered that SOTA methods diverge notably from compute-optimal setups, using excess compute for their level of robustness. Leveraging a compute-efficient setup, we surpass the prior SOTA with 20% (70%) fewer training (inference) FLOPs. We trained various compute-efficient models, with our best achieving 74% AutoAttack accuracy (+3% gain). However, our scaling laws also predict robustness slowly grows then plateaus at 90%: dwarfing our new SOTA by scaling is impractical, and perfect robustness is impossible. To better understand this predicted limit, we carry out a small-scale human evaluation on the AutoAttack data that fools our top-performing model. Concerningly, we estimate that human performance also plateaus near 90%, which we show to be attributable to ℓ∞-constrained attacks' generation of invalid images not consistent with their original labels. Having characterized limiting roadblocks, we outline promising paths for future research.