From Anomalies to Failures: Constructing Causal Error Graphs for Agentic Trace Diagnosis
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
LLM-driven agents are increasingly deployed in complex applications, where long agentic traces make failures difficult to diagnose. Existing trace diagnosis methods often conflate anomalies, errors, and failures, making diagnostic targets ambiguous; they also lack structured modeling of how causally relevant errors propagate and amplify into final task failures, resulting in unreliable failure attribution. To address these problems, we propose CEG-Agent, a tool-augmented agentic framework for causal diagnosis of agentic traces. Specifically, CEG-Agent introduces an explicit taxonomy of anomalies, errors, and failures, and constructs Causal Error Graphs (CEGs), a unified typed representation that links execution events, diagnostic nodes, and failure outcomes through causal relations. To evaluate causal trace diagnosis, we further construct CEG-Bench, a fully agent-annotated benchmark with high-confidence, consensus-derived CEG annotations obtained through an Adversarial Agentic Adjudication Protocol (AAAP). We validate the resulting annotations against an expert-curated human gold set, which shows close agreement with the automatic annotations. Experiments on CEG-Bench demonstrate that CEG-Agent achieves state-of-the-art performance under both semantically relaxed and structurally exact evaluation criteria. Our code is publicly available.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 16 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
15 of 16 references.
Closest ancestors · 1
Same direction, newest first
- 2026TraceSIR: A Multi-Agent Framework for Structured Analysis and Reporting of Agentic Execution Traces28 Feb
Other references · 14
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
- 2024Failure Modes of LLMs for Causal Reasoning on Narrativescited by 12
- 2026CodeTracer: Towards Traceable Agent Statescited by 11
- 2024Cascading Failure Prediction via Causal Inferencecited by 3
- 2025StepMathAgent: A Step-Wise Agent for Evaluating Mathematical Processes through Tree-of-Errorcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AgentTrace: Causal Graph Tracing for Root Cause Analysis in Deployed Multi-Agent Systems
- 2026AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents
- 2026SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation
- 2026Towards Self-Improving Error Diagnosis in Multi-Agent Systems
- 2026Adaptive Influence Graphs for Failure Attribution in Multi-Agent Systems
- 2026Diagnosing with Insights: Structured Analysis of Agent Failures via Behavioral Abstractions
- 2026Agent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training
- 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?