# FLARE: Agentic Coverage-Guided Fuzzing for LLM-Based Multi-Agent Systems

arXiv: https://arxiv.org/abs/2604.05289 · HTML: https://mlsanity.com/paper/2604.05289/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 60 references that are in the map.
- First posted on arXiv 7 Apr 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 14 of 14)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://mlsanity.com/paper/2412.14470/) (2024)
- [AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges](https://mlsanity.com/paper/2505.10468/) (2025)
- [Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey](https://mlsanity.com/paper/2402.09283/) (2024)
- [Experiential Co-Learning of Software-Developing Agents](https://mlsanity.com/paper/2312.17025/) (2023)
- [textitAgents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks](https://mlsanity.com/paper/2504.00218/) (2025)
- [HedgeAgents: A Balanced-aware Multi-agent Financial Trading System](https://mlsanity.com/paper/2502.13165/) (2025)
- [Doc2OracLL: Investigating the Impact of Documentation on LLM-based Test Oracle Generation](https://mlsanity.com/paper/2412.09360/) (2024)
- [Locating Faults with Program Slicing: An Empirical Analysis](https://mlsanity.com/paper/2101.03008/) (2021)
- [Motivations, Challenges, Best Practices, and Benefits for Bots and Conversational Agents in Software Engineering: A Multivocal Literature Review](https://mlsanity.com/paper/2409.11864/) (2024)
- [Technical Implementation of Tippy: Multi-Agent Architecture and System Design for Drug Discovery Laboratory Automation](https://mlsanity.com/paper/2507.17852/) (2025)
- [InfoSynth: Information-Guided Benchmark Synthesis for LLMs](https://mlsanity.com/paper/2601.00575/) (2026)

## Abstract (arXiv)

Multi-Agent LLM Systems (MAS) have been adopted to automate complex human workflows by breaking down tasks into subtasks. However, due to the non-deterministic behavior of LLM agents and the intricate interactions between agents, MAS applications frequently encounter failures, including infinite loops and failed tool invocations. Traditional software testing techniques are ineffective in detecting such failures due to the lack of LLM agent specification, the large behavioral space of MAS, and semantic-based correctness judgment. This paper presents FLARE, a novel testing framework tailored for MAS. FLARE takes the source code of MAS as input and extracts specifications and behavioral spaces from agent definitions. Based on these specifications, FLARE builds test oracles and conducts coverage-guided fuzzing to expose failures. It then analyzes execution logs to judge whether each test has passed and generates failure reports. Our evaluation on 16 diverse open-source applications demonstrates that FLARE achieves 96.9% inter-agent coverage and 91.1% intra-agent coverage, outperforming baselines by 9.5% and 1.0%. FLARE also uncovers 56 previously unknown failures unique to MAS.