# PersonaTeaming: Exploring How Introducing Personas Can Improve Automated AI Red-Teaming

arXiv: https://arxiv.org/abs/2509.03728 · HTML: https://mlsanity.com/paper/2509.03728/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 3 Sep 2025.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 25 of 25)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts](https://mlsanity.com/paper/2309.10253/) (2023)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts](https://mlsanity.com/paper/2402.16822/) (2024)
- [AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs](https://mlsanity.com/paper/2410.05295/) (2024)
- [Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming](https://mlsanity.com/paper/2501.18837/) (2025)
- [Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models](https://mlsanity.com/paper/2102.02503/) (2021)
- [Red-Teaming for Generative AI: Silver Bullet or Security Theater?](https://mlsanity.com/paper/2401.15897/) (2024)
- [Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors](https://mlsanity.com/paper/2105.02980/) (2021)
- [Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits](https://mlsanity.com/paper/2205.06922/) (2022)
- [Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice](https://mlsanity.com/paper/2210.03709/) (2022)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AI](https://mlsanity.com/paper/2501.01397/) (2025)
- [RainbowPlus: Enhancing Adversarial Prompt Generation via Evolutionary Quality-Diversity Search](https://mlsanity.com/paper/2504.15047/) (2025)
- [Ruby Teaming: Improving Quality Diversity Search with Memory for Automated Red Teaming](https://mlsanity.com/paper/2406.11654/) (2024)
- [Ferret: Faster and Effective Automated Red Teaming with Reward-Based Scoring Technique](https://mlsanity.com/paper/2408.10701/) (2024)
- [Effective Automation to Support the Human Infrastructure in AI Red Teaming](https://mlsanity.com/paper/2503.22116/) (2025)

## Built on by (most cited first, 9 of 9)

- [Doubly-Robust LLM-as-a-Judge: Externally Valid Estimation with Imperfect Personas](https://mlsanity.com/paper/2509.22957/) (2025)
- [Persona Generators: Generating Diverse Synthetic Personas for Arbitrary Contexts](https://mlsanity.com/paper/2602.03545/) (2026)
- [German General Social Survey Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studies](https://mlsanity.com/paper/2511.21722/) (2025)
- [The Persona Paradox: Medical Personas as Behavioral Priors in Clinical Language Models](https://mlsanity.com/paper/2601.05376/) (2026)
- [Personalities at Play: Probing Alignment in AI Teammates](https://mlsanity.com/paper/2603.00429/) (2026)
- [Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignment](https://mlsanity.com/paper/2605.01899/) (2026)
- [PolicyPad: Collaborative Prototyping of LLM Policies](https://mlsanity.com/paper/2509.19680/) (2025)
- [Position: AI Governance Needs ISO-like Interoperability Protocols, Not Just Laws](https://mlsanity.com/paper/2608.14568/) (2026)
- [Extracting Persona Subspaces Through Iterative Nullspace Projection For Modulation](https://mlsanity.com/paper/2610.04676/) (2026)

## Abstract (arXiv)

Recent developments in AI governance and safety research have called for red-teaming methods that can effectively surface potential risks posed by AI models. Many of these calls have emphasized how the identities and backgrounds of red-teamers can shape their red-teaming strategies, and thus the kinds of risks they are likely to uncover. While automated red-teaming approaches promise to complement human red-teaming by enabling larger-scale exploration of model behavior, current approaches do not consider the role of identity. As an initial step towards incorporating people's background and identities in automated red-teaming, we develop and evaluate a novel method, PersonaTeaming, that introduces personas in the adversarial prompt generation process to explore a wider spectrum of adversarial strategies. In particular, we first introduce a methodology for mutating prompts based on either "red-teaming expert" personas or "regular AI user" personas. We then develop a dynamic persona-generating algorithm that automatically generates various persona types adaptive to different seed prompts. In addition, we develop a set of new metrics to explicitly measure the "mutation distance" to complement existing diversity measurements of adversarial prompts. Our experiments show promising improvements (up to 144.1%) in the attack success rates of adversarial prompts through persona mutation, while maintaining prompt diversity, compared to RainbowPlus, a state-of-the-art automated red-teaming method. We discuss the strengths and limitations of different persona types and mutation methods, shedding light on future opportunities to explore complementarities between automated and human red-teaming approaches.