mlsanityData to 14 Oct 2026

AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

arXiv 2506.14682 ↗First posted 17 Jun 2025cs.CRcited by 7

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

We introduce AIRTBench, an AI red teaming benchmark for evaluating language models' ability to autonomously discover and exploit Artificial Intelligence and Machine Learning (AI/ML) security vulnerabilities. The benchmark consists of 70 realistic black-box capture-the-flag (CTF) challenges from the Crucible challenge environment on the Dreadnode platform, requiring models to write python code to interact with and compromise AI systems. Claude-3.7-Sonnet emerged as the clear leader, solving 43 challenges (61% of the total suite, 46.9% overall success rate), with Gemini-2.5-Pro following at 39 challenges (56%, 34.3% overall), GPT-4.5-Preview at 34 challenges (49%, 36.9% overall), and DeepSeek R1 at 29 challenges (41%, 26.9% overall). Our evaluations show frontier models excel at prompt injection attacks (averaging 49% success rates) but struggle with system exploitation and model inversion challenges (below 26%, even for the best performers). Frontier models are far outpacing open-source alternatives, with the best truly open-source model (Llama-4-17B) solving 7 challenges (10%, 1.0% overall), though demonstrating specialized capabilities on certain hard challenges. Compared to human security researchers, large language models (LLMs) solve challenges with remarkable efficiency completing in minutes what typically takes humans hours or days-with efficiency advantages of over 5,000x on hard challenges. Our contribution fills a critical gap in the evaluation landscape, providing the first comprehensive benchmark specifically designed to measure and track progress in autonomous AI red teaming capabilities.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 18 of its 29 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

18 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025The Automation Advantage in AI Red Teaming28 Apr

Other references · 17

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  7. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  8. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 421
  9. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 238
  10. 2024IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilitiescited by 103
  11. 2024NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Securitycited by 93
  12. 2024EnIGMA: Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilitiescited by 52
5 more
  1. 2024LLM4Decompile: Decompiling Binary Code with Large Language Modelscited by 39
  2. 2024An Empirical Evaluation of LLMs for Solving Offensive Security Challengescited by 35
  3. 2025AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defensescited by 15
  4. 2024Optimizing Large Language Model Hyperparameters for Code Generationcited by 11
  5. 2024Hierarchical Prompting Taxonomy: A Universal Evaluation Framework for Large Language Models Aligned with Human Cognitive Principlescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025PoCo: Agentic Proof-of-Concept Exploit Generation for Smart Contractscited by 8
  2. 2025PentestJudge: Judging Agent Behavior Against Operational Requirementscited by 7
  3. 2025Evaluating LLM Generated Detection Rules in Cybersecuritycited by 4
  4. 2025Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Securitycited by 2
  5. 2026Redefining AI Red Teaming in the Agentic Era: From Weeks to Hourscited by 1
  6. 2026Terminal Agents: A Survey of AI Agents in Command-Line Environmentscited by 1
  7. 2026A Translational Note on AI Safety Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026RedBench: A Universal Dataset for Comprehensive Red Teaming of Large Language Models
  2. 2024Automated Progressive Red Teaming
  3. 2026Red-TTT: Test-Time Training for Automated Jailbreaking Large Language Models
  4. 2025Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
  6. 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Tester
  7. 2025Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition
  8. 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs