# DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems

arXiv: https://arxiv.org/abs/2512.06749 · HTML: https://mlsanity.com/paper/2512.06749/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 25 of its 37 references that are in the map.
- First posted on arXiv 7 Dec 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 25 of 25)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [Agent-as-a-Judge: Evaluate Agents with Agents](https://mlsanity.com/paper/2410.10934/) (2024)
- [Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks](https://mlsanity.com/paper/2411.04468/) (2024)
- [Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems](https://mlsanity.com/paper/2505.00212/) (2025)
- [GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers](https://mlsanity.com/paper/2402.19255/) (2024)
- [AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?](https://mlsanity.com/paper/2407.15711/) (2024)
- [AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?](https://mlsanity.com/paper/2509.03312/) (2025)
- [TRAIL: Trace Reasoning and Agentic Issue Localization](https://mlsanity.com/paper/2505.08638/) (2025)
- [MathChat: Converse to Tackle Challenging Math Problems with LLM Agents](https://mlsanity.com/paper/2306.01337/) (2023)
- [Interactive Debugging and Steering of Multi-Agent AI Systems](https://mlsanity.com/paper/2503.02068/) (2025)
- [Evaluating Agent-based Program Repair at Google](https://mlsanity.com/paper/2501.07531/) (2025)
- [Who is Introducing the Failure? Automatically Attributing Failures of Multi-Agent Systems via Spectrum Analysis](https://mlsanity.com/paper/2509.13782/) (2025)
- [Where Did It All Go Wrong? A Hierarchical Look into Multi-Agent Error Attribution](https://mlsanity.com/paper/2510.04886/) (2025)
- [GraphTracer: Graph-Guided Failure Tracing in LLM Agents for Robust Multi-Turn Deep Search](https://mlsanity.com/paper/2510.10581/) (2025)
- [Automatic Failure Attribution and Critical Step Prediction Method for Multi-Agent Systems Based on Causal Inference](https://mlsanity.com/paper/2509.08682/) (2025)
- [Assessing and Verifying Task Utility in LLM-Powered Applications](https://mlsanity.com/paper/2405.02178/) (2024)
- [Abduct, Act, Predict: Scaffolding Causal Inference for Automated Failure Attribution in Multi-Agent Systems](https://mlsanity.com/paper/2509.10401/) (2025)
- [Can Agents Fix Agent Issues?](https://mlsanity.com/paper/2505.20749/) (2025)

## Built on by (most cited first, 24 of 24)

- [From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws](https://mlsanity.com/paper/2606.06324/) (2026)
- [AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces](https://mlsanity.com/paper/2608.23041/) (2026)
- [Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2605.14892/) (2026)
- [FinEvolveBench: A Benchmark for Self-Evolving Agents on Low-Repetition Tasks with Implicit Rewards](https://mlsanity.com/paper/2606.06960/) (2026)
- [AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems](https://mlsanity.com/paper/2609.08572/) (2026)
- [MASRubric: Auditing Information Flow in Multi-Agent Systems with Failure-Distilled Pitfall Rubrics](https://mlsanity.com/paper/2602.23258/) (2026)
- [Retrieval-Conditioned Topology Selection with Provable Budget Conservation for Multi-Agent Code Generation](https://mlsanity.com/paper/2605.05657/) (2026)
- [Insights Generator: Systematic Corpus-Level Trace Diagnostics for LLM Agents](https://mlsanity.com/paper/2605.21347/) (2026)
- [AgenticRepair: Multi-Faceted Program Context Engineering for Agentic Vulnerability Repair](https://mlsanity.com/paper/2607.29422/) (2026)
- [ASCon: A Direction-Aware Reciprocal Agent--Step Contextualization Model for Failure Attribution in Multi-Agent Systems](https://mlsanity.com/paper/2608.10646/) (2026)
- [What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction](https://mlsanity.com/paper/2609.05882/) (2026)
- [Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution](https://mlsanity.com/paper/2605.27621/) (2026)
- [POIROT: Interrogating Agents for Failure Detection in Multi-Agent Systems](https://mlsanity.com/paper/2606.02282/) (2026)
- [Diagnosis-Driven Automatic Repair for Agentic Workflow via Symbolic Inference](https://mlsanity.com/paper/2607.02882/) (2026)
- [Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agents](https://mlsanity.com/paper/2607.11433/) (2026)
- [Localizing Emergent Failures in Agentic AI: Recovering Minimal Repair Families via Counterfactual Replay](https://mlsanity.com/paper/2608.29228/) (2026)
- [Chronicle: Cut-Point Replay for Regression Testing of LLM Agents](https://mlsanity.com/paper/2609.20625/) (2026)
- [DENSE: Distilling Agent Trajectories into Evidence-Grounded Shortcut Trees for Self-Refinement](https://mlsanity.com/paper/2609.21423/) (2026)
- [ProCredit: From Outcome Rewards to Progress Credit in Agentic Reinforcement Learning](https://mlsanity.com/paper/2609.27532/) (2026)
- [DAAF: From Failure Localization to Editable System Assets in LLM Agents](https://mlsanity.com/paper/2609.32498/) (2026)
- [The Right Lesson at the Right Step: Deriving Control Updates for Self-Evolving Agents](https://mlsanity.com/paper/2609.34988/) (2026)
- [Where MLLMs Fail and Why: Causal Task Decomposition for Capability Failure Diagnosis](https://mlsanity.com/paper/2609.38851/) (2026)
- [When Harnesses Lose the Signal: Causal Evaluation of Recovery in LLM Agents](https://mlsanity.com/paper/2610.00372/) (2026)
- [CoEvoke: Evoking Multi-Agent Collaboration for Trajectory Collection](https://mlsanity.com/paper/2610.14021/) (2026)

## Abstract (arXiv)

Large language model (LLM)-based multi-agent systems are challenging to debug because failures often arise from long, branching interaction traces. The prevailing practice is to leverage LLMs for log-based failure localization, attributing errors to a specific agent and step. However, this paradigm has two key limitations: (i) log-only debugging lacks validation, producing untested hypotheses, and (ii) single-step or single-agent attribution is often ill-posed, as we find that multiple distinct interventions can independently repair the failed task. To address the first limitation, we introduce DoVer, an intervention-driven debugging framework, which augments hypothesis generation with active verification through targeted interventions (e.g., editing messages, altering plans). For the second limitation, rather than evaluating on attribution accuracy, we focus on measuring whether the system resolves the failure or makes quantifiable progress toward task success, reflecting a more outcome-oriented view of debugging. Within the Magnetic-One agent framework, on the datasets derived from GAIA and AssistantBench, DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses. DoVer also performs effectively on a different dataset (GSMPlus) and agent framework (AG2), where it recovers 49% of failed trials. These results highlight intervention as a practical mechanism for improving reliability in agentic systems and open opportunities for more robust, scalable debugging methods for LLM-based multi-agent systems. Project website and code will be available at https://aka.ms/DoVer.