# Synthesizing Multi-Agent Harnesses for Vulnerability Discovery

arXiv: https://arxiv.org/abs/2604.20801 · HTML: https://mlsanity.com/paper/2604.20801/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 41 references that are in the map.
- First posted on arXiv 22 Apr 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 11 of 11)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs](https://mlsanity.com/paper/2406.11695/) (2024)
- [Multi-agent Architecture Search via Agentic Supernet](https://mlsanity.com/paper/2502.04180/) (2025)
- [LLM Agents can Autonomously Hack Websites](https://mlsanity.com/paper/2402.06664/) (2024)
- [Teams of LLM Agents can Exploit Zero-Day Vulnerabilities](https://mlsanity.com/paper/2406.01637/) (2024)
- [NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security](https://mlsanity.com/paper/2406.05590/) (2024)

## Built on by (most cited first, 13 of 13)

- [From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws](https://mlsanity.com/paper/2606.06324/) (2026)
- [AgentCyberRange: Benchmarking Frontier AI Systems in Realistic Cyber Ranges](https://mlsanity.com/paper/2606.14295/) (2026)
- [Self-Evolving Agents as Dynamic Graph Transformation: A Survey and New Perspective](https://mlsanity.com/paper/2608.18104/) (2026)
- [VideoHarness-RSI: Recursive Harness Self-Improvement for Long-Video Understanding with Frozen Vision-Language Models](https://mlsanity.com/paper/2608.24302/) (2026)
- [Antares: Foundation Models for Agentic Vulnerability Localization](https://mlsanity.com/paper/2608.02407/) (2026)
- [Revelio: Cost-Efficient Agentic Memory Safety Vulnerability Detection For Repository-Scale Codebases](https://mlsanity.com/paper/2606.22263/) (2026)
- [MESA:Task-Adaptive Multi-Structure Evidence Selection for Long-Horizon Agent Memory](https://mlsanity.com/paper/2608.10108/) (2026)
- [AI Security Policy Should Assess Systems, Not Only Models](https://mlsanity.com/paper/2605.09504/) (2026)
- [Red-Teaming the Agentic Red-Team](https://mlsanity.com/paper/2606.24496/) (2026)
- [Understanding Implicit Trust Errors in Core Carrier Networks through Multi-Agent Flaw Discovery and Analysis](https://mlsanity.com/paper/2607.10315/) (2026)
- [SelfOp: An Optimization Algorithm for Self-Improving Security Agents](https://mlsanity.com/paper/2609.22792/) (2026)
- [AECP: Artifact-Exclusive Communication Protocol for Multi-Agent Code Generation](https://mlsanity.com/paper/2610.06481/) (2026)
- [Evolve Together, Fall into Place: How Social Scaffolding Shapes Self-Evolving Multi-Agent Systems](https://mlsanity.com/paper/2610.19799/) (2026)

## Abstract (arXiv)

LLM agents have begun to find real security vulnerabilities that human auditors and automated fuzzers missed for decades, in source-available targets where the analyst can build and instrument the code. In practice the work is split among several agents, wired together by a harness: the program that fixes which roles exist, how they pass information, which tools each may call, and how retries are coordinated. When the language model is held fixed, changing only the harness can still change success rates by several-fold on public agent benchmarks, yet most harnesses are written by hand; recent harness optimizers each search only a narrow slice of the design space and rely on coarse pass/fail feedback that gives no diagnostic signal about why a trial failed. AgentFlow addresses both limitations with a typed graph DSL whose search space jointly covers agent roles, prompts, tools, communication topology, and coordination protocol, paired with a feedback-driven outer loop that reads runtime signals from the target program itself to diagnose which part of the harness caused the failure and rewrite it accordingly. We evaluate AgentFlow on TerminalBench-2 with Claude Opus 4.6 and on Google Chrome with Kimi K2.5. AgentFlow reaches 84.3% on TerminalBench-2, the highest score in the public leaderboard snapshot we evaluate against, and discovers ten previously unknown zero-day vulnerabilities in Google Chrome, including two Critical sandbox-escape vulnerabilities (CVE-2026-5280 and CVE-2026-6297).