Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes
On the map: LLM reasoning → Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
Agentic AI systems combine LLM-based reasoning, orchestration, tool invocation, and interaction with external environments. These systems introduce faults that are difficult to characterize using existing taxonomies. To address this gap, we present an empirical study of faults in agentic AI systems. We collected 13,602 issues and pull requests from 40 repositories and, using stratified sampling, selected 385 faults for analysis. Through grounded theory, we derived taxonomies of fault types, symptoms, and root causes. We then used Apriori-based association rule mining to identify relationships among faults, symptoms, and root causes, and validated the taxonomy through a developer study with 145 practitioners. Our analysis produced a taxonomy of 34 fault types, organized into four architectural dimensions. These faults manifested as failures in structured-output interpretation, tool calls, runtime execution, and exception handling, with root causes including data schema mismatches, dependency drift, state management complexity, and model interface instability. Furthermore, association rules showed recurring cross-component propagation, linking structured data, dependency, and state management faults to their symptoms and root causes. Practitioners considered the taxonomy representative of agentic AI failures and suggested refinements related to multi-agent coordination and observability. These findings provide an empirical basis for diagnosing faults and improving reliability in agentic AI systems.
Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 16 of its 53 references that are in the map. First posted on arXiv 6 Mar 2026.
Builds on
16 of 53 references.
References · 16
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
- 2016Concrete Problems in AI Safetycited by 1,642
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
- 2025Where LLM Agents Fail and How They can Learn From Failurescited by 133
- 2024LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigationcited by 93
- 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistantcited by 54
- 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 50
- 2024AgentOps: Enabling Observability of LLM Agentscited by 46
- 2025Exploring Autonomous Agents: A Closer Look at Why They Fail When Completing Taskscited by 20
- 2022Bugs in Machine Learning-based Systems: A Faultload Benchmarkcited by 15
- 2025An Empirical Study of Testing Practices in Open Source AI Agent Frameworks and Agentic Applicationscited by 15
4 more
- 2023Bug Characterization in Machine Learning-based Systemscited by 10
- 2024Towards Enhancing the Reproducibility of Deep Learning Bugs: An Empirical Studycited by 6
- 2024Towards Understanding the Impact of Data Bugs on Deep Learning Models in Software Engineeringcited by 6
- 2020Are Multi-language Design Smells Fault-prone? An Empirical Studycited by 5
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failurescited by 10
- 2026CUADebug: Diagnosing and Repairing Computer-Use Agent Failurescited by 4
- 2026Understanding Agent-Reactive Bugs at the Model-Harness Boundary: An Empirical Study of LLM Agent Issue Reportscited by 3
- 2026Repair or Resample? Rethinking Failure Debugging in LLM Multi-Agent Systemscited by 3
- 2026Beyond State Machines: Executing Network Procedures with Agentic Tool-Calling Sequencescited by 2
- 2026Do Agent Benchmarks Do What They Say? An Executable-Contract Audit of Tool-Using Agent Environmentscited by 2
- 2026The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Projectcited by 1
- 2026SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Executioncited by 1
- 2026Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoningcited by 1
- 2026Benchmarks are Not Enough: RAMP for Runtime Assessing of Agentic Models in Production Systemscited by 0
- 2026LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designscited by 0
- 2026Explaining AI Agents Through Execution Tracescited by 0
- 2026Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning29 Sep 2026
- 2026Do Agent Benchmarks Do What They Say? An Executable-Contract Audit of Tool-Using Agent Environments29 Sep 2026
- 2026Explaining AI Agents Through Execution Traces5 Sep 2026
- 2026Repair or Resample? Rethinking Failure Debugging in LLM Multi-Agent Systems26 Aug 2026
- 2026SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Execution6 Aug 2026
- 2026LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs4 Aug 2026
- 2026CUADebug: Diagnosing and Repairing Computer-Use Agent Failures31 Jul 2026
- 2026Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures30 Jul 2026
- 2026Understanding Agent-Reactive Bugs at the Model-Harness Boundary: An Empirical Study of LLM Agent Issue Reports17 Jul 2026
- 2026Benchmarks are Not Enough: RAMP for Runtime Assessing of Agentic Models in Production Systems26 May 2026
- 2026Beyond State Machines: Executing Network Procedures with Agentic Tool-Calling Sequences4 May 2026
- 2026The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Project22 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Understanding Bugs in Modern Agentic Frameworks: A Study of Symptoms, Root Causes, and Triggering Conditions
- 2024Defining and Detecting the Defects of the Large Language Model-based Autonomous Agents
- 2026Reproducible, Explainable, and Effective Evaluations of Agentic AI for Software Engineering
- 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?
- 2026MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems
- 2025Agentic AI Process Observability: Discovering Behavioral Variability
- 2026Why Agentic Workflows Fail: A Taxonomy Derived from Literature and Empirical Data
- 2026When Agentic Executions Fail: Detecting and Localizing Runtime Faults from Telemetry