mlsanityData to 14 Oct 2026

AutoRed: A Free-form Adversarial Prompt Generation Framework for Automated Red Teaming

arXiv 2510.08329 ↗First posted 9 Oct 2025cs.CLcited by 1

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

The safety of Large Language Models (LLMs) is crucial for the development of trustworthy AI applications. Existing red teaming methods often rely on seed instructions, which limits the semantic diversity of the synthesized adversarial prompts. We propose AutoRed, a free-form adversarial prompt generation framework that removes the need for seed instructions. AutoRed operates in two stages: (1) persona-guided adversarial instruction generation, and (2) a reflection loop to iteratively refine low-quality prompts. To improve efficiency, we introduce a verifier to assess prompt harmfulness without querying the target models. Using AutoRed, we build two red teaming datasets -- AutoRed-Medium and AutoRed-Hard -- and evaluate eight state-of-the-art LLMs. AutoRed achieves higher attack success rates and better generalization than existing baselines. Our results highlight the limitations of seed-based approaches and demonstrate the potential of free-form red teaming for LLM safety evaluation. We will open source our datasets in the near future.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 38 of its 47 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

38 of 47 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Curiosity-driven Red-teaming for Large Language Models29 Feb
  2. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts26 Feb
  3. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal6 Feb
  4. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teaming13 Nov
  5. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment18 Aug
  6. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned23 Aug

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  8. 2024Qwen2 Technical Reportcited by 2,120
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  10. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  11. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  12. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
20 more
  1. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  2. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  3. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  4. 2024A StrongREJECT for Empty Jailbreakscited by 385
  5. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 288
  6. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
  7. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 194
  8. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  9. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 156
  10. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 138
  11. 2024Benchmarking Benchmark Leakage in Large Language Modelscited by 106
  12. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 98
  13. 2024CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Modelscited by 76
  14. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialoguecited by 66
  15. 2024EasyJailbreak: A Unified Framework for Jailbreaking Large Language Modelscited by 60
  16. 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Typescited by 60
  17. 2022CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Modelscited by 53
  18. 2023How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilitiescited by 33
  19. 2025SaRO: Enhancing LLM Safety through Reasoning-based Alignmentcited by 26
  20. 2025Can You Really Trust Code Copilots? Evaluating Large Language Models from a Code Security Perspectivecited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026FinRT: Distilling Adaptive Red-Teaming Strategies into Reusable Adversarial Generators in Consumer Financecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Automated Progressive Red Teaming
  2. 2025Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models
  3. 2026AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability
  4. 2026Training a General Purpose Automated Red Teaming Model
  5. 2026REPRO: Reflective Evolutionary Prompt Optimizer for Automated LLM Red-Teaming
  6. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Models
  7. 2025Jailbreak-Zero: A Path to Pareto Optimal Red Teaming for Large Language Models
  8. 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Tester