mlsanityData to 7 Oct 2026

MultiRobustBench: Benchmarking Robustness Against Multiple Attacks

arXiv 2302.10980 ↗First posted 21 Feb 2023cs.LG cs.CRcited by 7

Not on the map. Why?

Abstract (arXiv)

The bulk of existing research in defending against adversarial examples focuses on defending against a single (typically bounded Lp-norm) attack, but for a practical setting, machine learning (ML) models should be robust to a wide variety of attacks. In this paper, we present the first unified framework for considering multiple attacks against ML models. Our framework is able to model different levels of learner's knowledge about the test-time adversary, allowing us to model robustness against unforeseen attacks and robustness against unions of attacks. Using our framework, we present the first leaderboard, MultiRobustBench, for benchmarking multiattack evaluation which captures performance across attack types and attack strengths. We evaluate the performance of 16 defended models for robustness against a set of 9 different attack types, including Lp-based threat models, spatial transformations, and color changes, at 20 different attack strengths (180 attacks total). Additionally, we analyze the state of current defenses against multiple attacks. Our analysis shows that while existing defenses have made progress in terms of average robustness across the set of attacks used, robustness against the worst-case attack is still a big open problem as all existing models perform worse than random guessing.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 25 of its 41 references that are in the map. First posted on arXiv 21 Feb 2023.

Builds on

25 of 41 references.

References · 25

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  5. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  6. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  7. 2018Robustness May Be at Odds with Accuracycited by 873
  8. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  9. 2020Overfitting in adversarially robust deep learningcited by 504
  10. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  11. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  12. 2021Improving Robustness using Generated Datacited by 233
13 more
  1. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
  2. 2018Spatially Transformed Adversarial Examplescited by 195
  3. 2019Adversarial Training and Robustness for Multiple Perturbationscited by 182
  4. 2020Perceptual Adversarial Robustness: Defense Against Unseen Threat Modelscited by 137
  5. 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
  6. 2019Adversarial Robustness Against the Union of Multiple Perturbation Modelscited by 88
  7. 2019Functional Adversarial Attackscited by 82
  8. 2020Do Wider Neural Networks Really Help Adversarial Robustness?cited by 79
  9. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
  10. 2021Mind the box: l1-APGD for sparse adversarial attacks on image classifierscited by 50
  11. 2019Provable robustness against all adversarial lp-perturbations for p≥ 1cited by 39
  12. 2022Towards Compositional Adversarial Robustness: Generalizing Adversarial Training to Composite Semantic Perturbationscited by 26
  13. 2022Formulating Robustness Against Unforeseen Attackscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 2; 2026: 1.

7 papers.

  1. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  2. 2023Adversarial Machine Learning for Social Good: Reframing the Adversary as an Allycited by 5
  3. 2023OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shiftcited by 4
  4. 2024Learning from Mistakes: Self-correct Adversarial Training for Chinese Unnatural Text Correctioncited by 2
  5. 2025Adapting to Evolving Adversaries with Regularized Continual Robust Trainingcited by 2
  6. 2025Stabilizing Multi-Attack Adversarial Training via Bandit Optimizationcited by 0
  7. 2026Auto-ART: Structured Literature Synthesis and Automated Adversarial Robustness Testingcited by 0