mlsanityData to 7 Oct 2026

A Systematic Review of Algorithmic Red Teaming Methodologies for Assurance and Security of AI Applications

arXiv 2602.21267 ↗First posted 24 Feb 2026cs.CR cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Cybersecurity threats are becoming increasingly sophisticated, making traditional defense mechanisms and manual red teaming approaches insufficient for modern organizations. While red teaming has long been recognized as an effective method to identify vulnerabilities by simulating real-world attacks, its manual execution is resource-intensive, time-consuming, and lacks scalability for frequent assessments. These limitations have driven the evolution toward auto-mated red teaming, which leverages artificial intelligence and automation to deliver efficient and adaptive security evaluations. This systematic review consolidates existing research on automated red teaming, examining its methodologies, tools, benefits, and limitations. The paper also highlights current trends, challenges, and research gaps, offering insights into future directions for improving automated red teaming as a critical component of proactive cybersecurity strategies. By synthesizing findings from diverse studies, this review aims to provide a comprehensive understanding of how automation enhances red teaming and strengthens organizational resilience against evolving cyber threats.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 59 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

34 of 59 references.

References · 34

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  3. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
  4. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 208
  5. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 206
  6. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 189
  7. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  8. 2025LlamaFirewall: An open source guardrail system for building secure AI agentscited by 114
  9. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
  10. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 93
  11. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  12. 2024A Cross-Language Investigation into Jailbreak Attacks in Large Language Modelscited by 37
22 more
  1. 2024S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Modelscited by 28
  2. 2025Lessons From Red Teaming 100 Generative AI Productscited by 22
  3. 2025Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judgescited by 19
  4. 2025T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attackscited by 18
  5. 2023MITRE ATT&CK: State of the Art and Way Forwardcited by 12
  6. 2025CySecBench: Generative AI-based CyberSecurity-focused Prompt Dataset for Benchmarking Large Language Modelscited by 10
  7. 2025MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attackscited by 8
  8. 2025LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasurescited by 8
  9. 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Modelscited by 8
  10. 2025ASTRAL: Automated Safety Testing of Large Language Modelscited by 7
  11. 2025Adversarial Prompt Evaluation: Systematic Benchmarking of Guardrails Against Prompt Input Attacks on LLMscited by 7
  12. 2025Securing Agentic AI: Threat Modeling and Risk Analysis for Network Monitoring Agentic AI Systemcited by 6
  13. 2025Dynamic Optimization and Safety Indicator Injection for Jailbreaking Text-to-Image Models with Multimodal Safety Filterscited by 5
  14. 2025JailBench: A Comprehensive Chinese Security Assessment Benchmark for Large Language Modelscited by 4
  15. 2025Security Steerability is All You Needcited by 4
  16. 2025GenBreak: Red Teaming Text-to-Image Generators Using Large Language Modelscited by 4
  17. 2025Effective Automation to Support the Human Infrastructure in AI Red Teamingcited by 3
  18. 2025The Automation Advantage in AI Red Teamingcited by 3
  19. 2025ARMs: Adaptive Red-Teaming Agent against Multimodal Models with Plug-and-Play Attackscited by 3
  20. 2025Anecdoctoring: Automated Red-Teaming Across Language and Placecited by 2
  21. 2025SecReEvalBench: A Multi-turned Security Resilience Evaluation Benchmark for Large Language Modelscited by 1
  22. 2025SoK: A Comprehensive Security Analysis of Jailbreak Resilience in GPT and DeepSeek Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Modelscited by 0