CORRECT: COndensed eRror RECognition via knowledge Transfer in multi-agent systems
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Multi-agent systems (MAS) are increasingly capable of tackling complex real-world tasks, yet their reliance on inter-agent coordination, tool use, and long-horizon reasoning makes error recognition particularly challenging. Minor errors can propagate across agents, escalating into task failures while producing long, intertwined execution trajectories that impose significant costs for both human developers and automated systems to debug and analyze. Our key insight is that, despite surface differences in failure trajectories (e.g., logs), MAS errors often recur with similar structural patterns. This paper presents CORRECT, the first lightweight, training-free framework that leverages an online cache of distilled error schemata to recognize and transfer knowledge of failure structures across new requests. This cache-based reuse allows LLMs to perform targeted error localization at inference time, avoiding the need for expensive retraining while adapting to dynamic MAS deployments in subseconds. To support rigorous study in this domain, we also introduce CORRECT-Error, a large-scale dataset of over 2,000 annotated trajectories collected through a novel error-injection pipeline guided by real-world distributions, and further validated through human evaluation to ensure alignment with natural failure patterns. Experiments across seven diverse MAS applications show that CORRECT improves step-level error localization up to 19.8% over existing advances while at near-zero overhead, substantially narrowing the gap between automated and human-level error recognition.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 36 of its 44 references that are in the map. First posted on arXiv 28 Sep 2025.
Builds on
36 of 44 references.
Closest ancestors · 6
Same direction, newest first
- 2025Who is Introducing the Failure? Automatically Attributing Failures of Multi-Agent Systems via Spectrum Analysis17 Sep
- 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems30 Apr
- 2025Why Do Multi-Agent LLM Systems Fail?17 Mar
- 2025Interactive Debugging and Steering of Multi-Agent AI Systems3 Mar
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug
- 2023ChatDev: Communicative Agents for Software Development16 Jul
Other references · 30
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2024GPT-4o System Cardcited by 4,467
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023Let's Verify Step by Stepcited by 3,598
18 more
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
- 2023GAIA: a benchmark for General AI Assistantscited by 1,084
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
- 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
- 2023Instruction Tuning with GPT-4cited by 676
- 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
- 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
- 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
- 2025SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoningcited by 80
- 2023ReAcTable: Enhancing ReAct for Table Question Answeringcited by 76
- 2025Single-agent or Multi-agent Systems? Why Not Both?cited by 28
- 2025Step-wise Adaptive Integration of Supervised Fine-tuning and Reinforcement Learning for Task-Specific LLMscited by 25
- 2025IC-Cache: Efficient Large Language Model Serving via In-context Cachingcited by 13
Built on by
Citing papers per year: 2025: 2; 2026: 12.
14 papers.
Sort:
By area:
- 2025CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimizationcited by 24
- 2025XRPO: Pushing the limits of GRPO with Targeted Exploration and Exploitationcited by 17
- 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 11
- 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 5
- 2026Conformal Agent Error Attributioncited by 3
- 2026The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Projectcited by 1
- 2026SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigationcited by 1
- 2026LongRCA Bench: Root-Cause Localization in Long-Horizon Agent Trajectoriescited by 1
- 2026AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systemscited by 1
- 2026Diagnosis-Driven Automatic Repair for Agentic Workflow via Symbolic Inferencecited by 0
- 2026Beyond LLM-Based Reasoning: Lightweight GNNs for Agent Failure Attributioncited by 0
- 2026Detect Before You Attribute: Cascade Failure Attribution for Multi-Agent Systemscited by 0
- 2026The Decomposition Tax: LLM Pipelines Lose Up to 40 Accuracy Points at Their Own Interfaces26 Sep 2026
- 2026Re-derivability Decides What a Staged Agent Pipeline Recovers After an Upstream Fault26 Sep 2026
- 2026AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems8 Sep 2026
- 2026Detect Before You Attribute: Cascade Failure Attribution for Multi-Agent Systems30 Aug 2026
- 2026Beyond LLM-Based Reasoning: Lightweight GNNs for Agent Failure Attribution19 Aug 2026
- 2026LongRCA Bench: Root-Cause Localization in Long-Horizon Agent Trajectories15 Aug 2026
- 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?10 Jul 2026
- 2026Diagnosis-Driven Automatic Repair for Agentic Workflow via Symbolic Inference3 Jul 2026
- 2026SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation23 Jun 2026
- 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems14 May 2026
- 2026Conformal Agent Error Attribution7 May 2026
- 2026The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Project22 Mar 2026
- 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 11
- 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 5
- 2026Conformal Agent Error Attributioncited by 3
- 2026The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Projectcited by 1
- 2026SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigationcited by 1
- 2026AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systemscited by 1
- 2026Diagnosis-Driven Automatic Repair for Agentic Workflow via Symbolic Inferencecited by 0
- 2026Detect Before You Attribute: Cascade Failure Attribution for Multi-Agent Systemscited by 0
- 2026Re-derivability Decides What a Staged Agent Pipeline Recovers After an Upstream Faultcited by 0
- 2026Re-derivability Decides What a Staged Agent Pipeline Recovers After an Upstream Fault26 Sep 2026
- 2026AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems8 Sep 2026
- 2026Detect Before You Attribute: Cascade Failure Attribution for Multi-Agent Systems30 Aug 2026
- 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?10 Jul 2026
- 2026Diagnosis-Driven Automatic Repair for Agentic Workflow via Symbolic Inference3 Jul 2026
- 2026SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation23 Jun 2026
- 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems14 May 2026
- 2026Conformal Agent Error Attribution7 May 2026
- 2026The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Project22 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Towards Self-Improving Error Diagnosis in Multi-Agent Systems
- 2026VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems
- 2026Agent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training
- 2025Where LLM Agents Fail and How They can Learn From Failures
- 2026StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems
- 2025Aegis: Automated Error Generation and Attribution for Multi-Agent Systems
- 2026Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems
- 2026TRAJDEBUG: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories