mlsanityData to 7 Oct 2026

Redefining AI Red Teaming in the Agentic Era: From Weeks to Hours

arXiv 2605.04019 ↗First posted 5 May 2026cs.AI cs.CRcited by 1

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

AI systems are entering critical domains like healthcare, finance, and defense, yet remain vulnerable to adversarial attacks. While AI red teaming is a primary defense, current approaches force operators into manual, library-specific workflows. Operators spend weeks hand-crafting workflows - assembling attacks, transforms, and scorers. When results fall short, workflows must be rebuilt. As a result, operators spend more time constructing workflows than probing targets for security and safety vulnerabilities. We introduce an AI red teaming agent built on the open-source Dreadnode SDK. The agent creates workflows grounded in 45+ adversarial attacks, 450+ transforms, and 130+ scorers. Operators can probe multi-agent systems, multilingual, and multimodal targets, focusing on what to probe rather than how to implement it. We make three contributions: 1. Agentic interface. Operators describe goals in natural language via the Dreadnode TUI (Terminal User Interface). The agent handles attack selection, transform composition, execution, and reporting, letting operators focus on red teaming. Weeks compress to hours. 2. Unified framework. A single framework for probing traditional ML models (adversarial examples) and generative AI systems (jailbreaks), removing the need for separate libraries. 3. Llama Scout case study. We red team Meta Llama Scout and achieve an 85% attack success rate with severity up to 1.0, using zero human-developed code

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 32 of its 49 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

32 of 49 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025PentestJudge: Judging Agent Behavior Against Operational Requirements4 Aug
  2. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 30

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  7. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  8. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2022Red Teaming Language Models with Language Modelscited by 983
  11. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  12. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
18 more
  1. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  2. 2017Adversarial Examples: Attacks and Defenses for Deep Learningcited by 350
  3. 2019HopSkipJumpAttack: A Query-Efficient Decision-Based Attackcited by 328
  4. 2023Are aligned neural networks adversarially aligned?cited by 305
  5. 2023Low-Resource Languages Jailbreak GPT-4cited by 290
  6. 2019Simple Black-box Adversarial Attackscited by 249
  7. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 189
  8. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  9. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  10. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 130
  11. 2025AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integrationcited by 38
  12. 2026Dive into Claude Code: The Design Space of Today's and Future AI Agent Systemscited by 33
  13. 2024Phi-3 Safety Post-Training: Aligning Language Models with a "Break-Fix" Cyclecited by 32
  14. 2024PyRIT: A Framework for Security Risk Identification and Red Teaming in Generative AI Systemcited by 29
  15. 2025Lessons From Red Teaming 100 Generative AI Productscited by 22
  16. 2025Graph of Attacks with Pruning: Optimizing Stealthy Jailbreak Prompt Generation for Enhanced LLM Content Moderationcited by 9
  17. 2025AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Modelscited by 7
  18. 2025The Automation Advantage in AI Red Teamingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Operational Evidence Gaps for LLMs in Fraud Detection and Trust-and-Safety Workflowscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition
  2. 2026Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming
  3. 2026REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems
  4. 2026AgenticRed: Evolving Agentic Systems for Red-Teaming
  5. 2026Governing Execution Risk in Agentic AI Systems: A Trajectory-Guided Framework for Red Teaming
  6. 2026RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems
  7. 2026Red-Teaming the Agentic Red-Team
  8. 2025Mind the Gap: Comparing Model- vs Agentic-Level Red Teaming with Action-Graph Observability on GPT-OSS-20B