mlsanityData to 7 Oct 2026

Understanding Bugs in Modern Agentic Frameworks: A Study of Symptoms, Root Causes, and Triggering Conditions

arXiv 2604.08906 ↗First posted 10 Apr 2026cs.SEcited by 1

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Modern agentic frameworks such as CrewAI and AutoGen have evolved into complex, autonomous multi-agent systems, introducing reliability challenges that go beyond earlier pipeline-based LLM libraries. However, existing empirical studies focus on earlier LLM libraries or task-level bugs, leaving the unique complexities of these agentic frameworks unexplored. We present a comprehensive study of 409 fixed bugs across five representative agentic frameworks, proposing a five-layer architectural abstraction. Our taxonomy identifies previously unreported symptom categories---Unexpected Execution Sequence, User Configuration Ignored, and Incomplete/Incorrect Trace---and isolates agent-specific root causes including Model-Related Fault, Cognitive Context Mismanagement, and Orchestration Fault. Notably, the model integration layer is the most bug-prone yet receives disproportionately low test inclusion rate during bug fixing (47%), revealing a critical validation gap. Despite varying design paradigms, bug symptoms, root causes, and bug-prone components show substantial cross-framework consistency (JS similarity 0.62--0.88). Finally, we present the first systematic study of bug-triggering conditions, identifying error-prone factor combinations across element configurations, input patterns, and operations, and demonstrate their transferability across frameworks, providing a foundation for test oracle design and cross-framework benchmark.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 6 of its 51 references that are in the map. First posted on arXiv 10 Apr 2026.

Builds on

6 of 51 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026When Agents Fail: A Comprehensive Study of Bugs in LLM Agents with Automated Labeling21 Jan

Other references · 5

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2025Exploring Autonomous Agents: A Closer Look at Why They Fail When Completing Taskscited by 20
  3. 2025A First Look at Bugs in LLM Inference Enginescited by 5
  4. 2026A Large-Scale Study on the Development and Issues of Multi-Agent AI Systemscited by 5
  5. 2024Understanding and Detecting Annotation-Induced Faults of Static Analyzerscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Understanding Agent-Reactive Bugs at the Model-Harness Boundary: An Empirical Study of LLM Agent Issue Reports
  2. 2026SelfHeal: Empirical Fix Pattern Analysis and Bug Repair in LLM Agents
  3. 2026Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes
  4. 2026Where Agent Frameworks Fall Short: Examining Functional Challenges and Usability Concerns
  5. 2026Automated structural testing of LLM-based agents: methods, framework, and case studies
  6. 2026MAESTRO: Multi-Agent Evaluation Suite for Testing, Reliability, and Observability
  7. 2026A Comprehensive Study of Implementation Bugs in Multi-modal Agents
  8. 2025Where LLM Agents Fail and How They can Learn From Failures