# FERRET: Framework for Expansion Reliant Red Teaming

arXiv: https://arxiv.org/abs/2603.10010 · HTML: https://mlsanity.com/paper/2603.10010/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 16 of its 16 references that are in the map.
- First posted on arXiv 17 Feb 2026.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 16 of 16)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack](https://mlsanity.com/paper/2404.01833/) (2024)
- [Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models](https://mlsanity.com/paper/2307.14539/) (2023)
- [MART: Improving LLM Safety with Multi-round Automatic Red-Teaming](https://mlsanity.com/paper/2311.07689/) (2023)
- [Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations](https://mlsanity.com/paper/2411.10414/) (2024)
- [Explore, Establish, Exploit: Red Teaming Language Models from Scratch](https://mlsanity.com/paper/2306.09442/) (2023)
- [FLIRT: Feedback Loop In-context Red Teaming](https://mlsanity.com/paper/2308.04265/) (2023)
- [Automated Red Teaming with GOAT: the Generative Offensive Agent Tester](https://mlsanity.com/paper/2410.01606/) (2024)
- [Gradient-Based Language Model Red Teaming](https://mlsanity.com/paper/2401.16656/) (2024)
- [DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrievers](https://mlsanity.com/paper/2502.18460/) (2025)
- [Emerging Vulnerabilities in Frontier Models: Multi-Turn Jailbreak Attacks](https://mlsanity.com/paper/2409.00137/) (2024)
- [Strategize Globally, Adapt Locally: A Multi-Turn Red Teaming Agent with Dual-Level Learning](https://mlsanity.com/paper/2504.01278/) (2025)
- [Red-Teaming Text-to-Image Systems by Rule-based Preference Modeling](https://mlsanity.com/paper/2505.21074/) (2025)

## Built on by (most cited first, 1 of 1)

- [EvoFlint: An Evolutionary Atlas of Multi-Turn LLM Vulnerabilities](https://mlsanity.com/paper/2609.00487/) (2026)

## Abstract (arXiv)

We introduce a multi-faceted automated red teaming framework in which the goal is to generate multi-modal adversarial conversations that would break a target model and introduce various expansions that would result in more effective and efficient adversarial conversations. The introduced expansions include: 1. Horizontal expansion in which the goal is for the red team model to self-improve and generate more effective conversation starters that would shape a conversation. 2. Vertical expansion in which the goal is to take these conversation starters that are discovered in the horizontal expansion phase and expand them into effective multi-modal conversations and 3. Meta expansion in which the goal is for the red team model to discover more effective multi-modal attack strategies during the course of a conversation. We call our framework FERRET (Framework for Expansion Reliant Red Teaming) and compare it with various existing automated red teaming approaches. In our experiments, we demonstrate the effectiveness of FERRET in generating effective multi-modal adversarial conversations and its superior performance against existing state of the art approaches.