# AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses

arXiv: https://arxiv.org/abs/2503.01811 · HTML: https://mlsanity.com/paper/2503.01811/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 71 of its 86 references that are in the map.
- First posted on arXiv 3 Mar 2025.

## Builds on (most cited first, 30 of 71)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://mlsanity.com/paper/1705.07263/) (2017)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality](https://mlsanity.com/paper/1801.02613/) (2018)
- [MagNet: a Two-Pronged Defense against Adversarial Examples](https://mlsanity.com/paper/1705.09064/) (2017)
- [FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI](https://mlsanity.com/paper/2411.04872/) (2024)
- [An Overview of Catastrophic AI Risks](https://mlsanity.com/paper/2306.12001/) (2023)
- [RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts](https://mlsanity.com/paper/2411.15114/) (2024)

## Built on by (most cited first, 15 of 15)

- [WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks](https://mlsanity.com/paper/2504.18575/) (2025)
- [CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale](https://mlsanity.com/paper/2506.02548/) (2025)
- [Cyber-Zero: Training Cybersecurity Agents without Runtime](https://mlsanity.com/paper/2508.00910/) (2025)
- [Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Models](https://mlsanity.com/paper/2509.01909/) (2025)
- [Training Language Model Agents to Find Vulnerabilities with CTF-Dojo](https://mlsanity.com/paper/2508.18370/) (2025)
- [Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing](https://mlsanity.com/paper/2512.09882/) (2025)
- [Dynamic Risk Assessments for Offensive Cybersecurity Agents](https://mlsanity.com/paper/2505.18384/) (2025)
- [AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models](https://mlsanity.com/paper/2506.14682/) (2025)
- [LLMs unlock new paths to monetizing exploits](https://mlsanity.com/paper/2505.11449/) (2025)
- [HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities](https://mlsanity.com/paper/2510.12200/) (2025)
- [Breaking Guardrails, Facing Walls: Insights on Adversarial AI for Defenders & Researchers](https://mlsanity.com/paper/2510.16005/) (2025)
- [Automated Membership Inference Attacks (AutoMIA): Discovering MIA Signal Computations using LLM Agents](https://mlsanity.com/paper/2603.19375/) (2026)
- [Auto-ART: Structured Literature Synthesis and Automated Adversarial Robustness Testing](https://mlsanity.com/paper/2604.20704/) (2026)
- [Engineering Trustworthy Agentic AI for Critical Systems](https://mlsanity.com/paper/2607.18548/) (2026)
- [CyberClear: A Benchmark for LLM Agent Systems on APT Attack Chain Provenance](https://mlsanity.com/paper/2609.32424/) (2026)

## Abstract (arXiv)

We introduce AutoAdvExBench, a benchmark to evaluate if large language models (LLMs) can autonomously exploit defenses to adversarial examples. Unlike existing security benchmarks that often serve as proxies for real-world tasks, bench directly measures LLMs' success on tasks regularly performed by machine learning security experts. This approach offers a significant advantage: if a LLM could solve the challenges presented in bench, it would immediately present practical utility for adversarial machine learning researchers. We then design a strong agent that is capable of breaking 75% of CTF-like ("homework exercise") adversarial example defenses. However, we show that this agent is only able to succeed on 13% of the real-world defenses in our benchmark, indicating the large gap between difficulty in attacking "real" code, and CTF-like code. In contrast, a stronger LLM that can attack 21% of real defenses only succeeds on 54% of CTF-like defenses. We make this benchmark available at https://github.com/ethz-spylab/AutoAdvExBench.