# Understanding Bugs in Modern Agentic Frameworks: A Study of Symptoms, Root Causes, and Triggering Conditions

arXiv: https://arxiv.org/abs/2604.08906 · HTML: https://mlsanity.com/paper/2604.08906/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 6 of its 51 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 6 of 6)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Exploring Autonomous Agents: A Closer Look at Why They Fail When Completing Tasks](https://mlsanity.com/paper/2508.13143/) (2025)
- [A Large-Scale Study on the Development and Issues of Multi-Agent AI Systems](https://mlsanity.com/paper/2601.07136/) (2026)
- [When Agents Fail: A Comprehensive Study of Bugs in LLM Agents with Automated Labeling](https://mlsanity.com/paper/2601.15232/) (2026)
- [A First Look at Bugs in LLM Inference Engines](https://mlsanity.com/paper/2506.09713/) (2025)
- [Understanding and Detecting Annotation-Induced Faults of Static Analyzers](https://mlsanity.com/paper/2402.14366/) (2024)

## Built on by (most cited first, 2 of 2)

- [AgentBug-Smith: Automatically Reproducing Real-World Harness Bugs in Agentic Systems](https://mlsanity.com/paper/2609.37864/) (2026)
- [Complex Agents, Shallow Tests: Demystifying and Enhancing Test Adequacy of Agent Harness in the Wild](https://mlsanity.com/paper/2610.04921/) (2026)

## Abstract (arXiv)

Modern agentic frameworks such as CrewAI and AutoGen have evolved into complex, autonomous multi-agent systems, introducing reliability challenges that go beyond earlier pipeline-based LLM libraries. However, existing empirical studies focus on earlier LLM libraries or task-level bugs, leaving the unique complexities of these agentic frameworks unexplored. We present a comprehensive study of 409 fixed bugs across five representative agentic frameworks, proposing a five-layer architectural abstraction. Our taxonomy identifies previously unreported symptom categories---Unexpected Execution Sequence, User Configuration Ignored, and Incomplete/Incorrect Trace---and isolates agent-specific root causes including Model-Related Fault, Cognitive Context Mismanagement, and Orchestration Fault. Notably, the model integration layer is the most bug-prone yet receives disproportionately low test inclusion rate during bug fixing (47%), revealing a critical validation gap. Despite varying design paradigms, bug symptoms, root causes, and bug-prone components show substantial cross-framework consistency (JS similarity 0.62--0.88). Finally, we present the first systematic study of bug-triggering conditions, identifying error-prone factor combinations across element configurations, input patterns, and operations, and demonstrate their transferability across frameworks, providing a foundation for test oracle design and cross-framework benchmark.