# A Systematic Review of Algorithmic Red Teaming Methodologies for Assurance and Security of AI Applications

arXiv: https://arxiv.org/abs/2602.21267 · HTML: https://mlsanity.com/paper/2602.21267/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 59 references that are in the map.
- First posted on arXiv 24 Feb 2026.

## Builds on (most cited first, 30 of 34)

- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts](https://mlsanity.com/paper/2309.10253/) (2023)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2404.03027/) (2024)
- [WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models](https://mlsanity.com/paper/2406.18510/) (2024)
- [SafetyBench: Evaluating the Safety of Large Language Models](https://mlsanity.com/paper/2309.07045/) (2023)
- [Security and Privacy Challenges of Large Language Models: A Survey](https://mlsanity.com/paper/2402.00888/) (2024)
- [The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections](https://mlsanity.com/paper/2510.09023/) (2025)
- [LlamaFirewall: An open source guardrail system for building secure AI agents](https://mlsanity.com/paper/2505.03574/) (2025)
- [A Survey on Trustworthy LLM Agents: Threats and Countermeasures](https://mlsanity.com/paper/2503.09648/) (2025)
- [Red-Teaming for Generative AI: Silver Bullet or Security Theater?](https://mlsanity.com/paper/2401.15897/) (2024)
- [A Cross-Language Investigation into Jailbreak Attacks in Large Language Models](https://mlsanity.com/paper/2401.16765/) (2024)
- [S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models](https://mlsanity.com/paper/2405.14191/) (2024)
- [Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judges](https://mlsanity.com/paper/2503.04474/) (2025)
- [Lessons From Red Teaming 100 Generative AI Products](https://mlsanity.com/paper/2501.07238/) (2025)
- [T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attacks](https://mlsanity.com/paper/2505.06679/) (2025)
- [MITRE ATT&CK: State of the Art and Way Forward](https://mlsanity.com/paper/2308.14016/) (2023)
- [CySecBench: Generative AI-based CyberSecurity-focused Prompt Dataset for Benchmarking Large Language Models](https://mlsanity.com/paper/2501.01335/) (2025)
- [MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attacks](https://mlsanity.com/paper/2503.19134/) (2025)
- [Adversarial Prompt Evaluation: Systematic Benchmarking of Guardrails Against Prompt Input Attacks on LLMs](https://mlsanity.com/paper/2502.15427/) (2025)
- [LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures](https://mlsanity.com/paper/2505.01177/) (2025)
- [AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models](https://mlsanity.com/paper/2507.01020/) (2025)
- [ASTRAL: Automated Safety Testing of Large Language Models](https://mlsanity.com/paper/2501.17132/) (2025)
- [Securing Agentic AI: Threat Modeling and Risk Analysis for Network Monitoring Agentic AI System](https://mlsanity.com/paper/2508.10043/) (2025)
- [JailBench: A Comprehensive Chinese Security Assessment Benchmark for Large Language Models](https://mlsanity.com/paper/2502.18935/) (2025)
- [Dynamic Optimization and Safety Indicator Injection for Jailbreaking Text-to-Image Models with Multimodal Safety Filters](https://mlsanity.com/paper/2505.18979/) (2025)
- [ARMs: Adaptive Red-Teaming Agent against Multimodal Models with Plug-and-Play Attacks](https://mlsanity.com/paper/2510.02677/) (2025)
- [Security Steerability is All You Need](https://mlsanity.com/paper/2504.19521/) (2025)
- [The Automation Advantage in AI Red Teaming](https://mlsanity.com/paper/2504.19855/) (2025)
- [GenBreak: Red Teaming Text-to-Image Generators Using Large Language Models](https://mlsanity.com/paper/2506.10047/) (2025)

## Built on by (most cited first, 1 of 1)

- [The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models](https://mlsanity.com/paper/2606.11082/) (2026)

## Abstract (arXiv)

Cybersecurity threats are becoming increasingly sophisticated, making traditional defense mechanisms and manual red teaming approaches insufficient for modern organizations. While red teaming has long been recognized as an effective method to identify vulnerabilities by simulating real-world attacks, its manual execution is resource-intensive, time-consuming, and lacks scalability for frequent assessments. These limitations have driven the evolution toward auto-mated red teaming, which leverages artificial intelligence and automation to deliver efficient and adaptive security evaluations. This systematic review consolidates existing research on automated red teaming, examining its methodologies, tools, benefits, and limitations. The paper also highlights current trends, challenges, and research gaps, offering insights into future directions for improving automated red teaming as a critical component of proactive cybersecurity strategies. By synthesizing findings from diverse studies, this review aims to provide a comprehensive understanding of how automation enhances red teaming and strengthens organizational resilience against evolving cyber threats.