Why Do AI Agents Systematically Fail at Cloud Root Cause Analysis?
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Failures in large-scale cloud systems incur substantial financial losses, making automated Root Cause Analysis (RCA) essential for operational stability. Recent efforts leverage Large Language Model (LLM) agents to automate this task, yet existing systems exhibit low detection accuracy even with capable models, and current evaluation frameworks assess only final answer correctness without revealing why the agent's reasoning failed. This paper presents a process level failure analysis of LLM-based RCA agents. We execute the full OpenRCA benchmark across five LLM models, producing 1,675 agent runs, and classify observed failures into 12 pitfall types across intra-agent reasoning, inter-agent communication, and agent-environment interaction. Our analysis reveals that the most prevalent pitfalls, notably hallucinated data interpretation and incomplete exploration, persist across all models regardless of capability tier, indicating that these failures originate from the shared agent architecture rather than from individual model limitations. Controlled mitigation experiments further show that prompt engineering alone cannot resolve the dominant pitfalls, whereas enriching the inter-agent communication protocol reduces communication-related failures by up to 15 percentage points. The pitfall taxonomy and diagnostic methodology developed in this work provide a foundation for designing more reliable autonomous agents for cloud RCA.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 6 of its 12 references that are in the map. First posted on arXiv 10 Feb 2026.
Builds on
6 of 12 references.
Closest ancestors · 1
Same direction, newest first
Other references · 5
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 223
- 2024RCAEval: A Benchmark for Root Cause Analysis of Microservice Systems with Telemetry Datacited by 26
- 2025Beyond Context: Large Language Models' Failure to Grasp Users' Intentcited by 1
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026CUJBench: Benchmarking LLM-Agent on Cross-Modal Failure Diagnosis from Browser to Backendcited by 1
- 2026Pooled Leaderboards Hide System-Specific Winners: A Reporting-Protocol Audit of Offline Root-Cause Analysis Benchmarkscited by 1
- 2026RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enoughcited by 1
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Modelcited by 1
- 2026OpsLLM: Construction of Large Language Model for Software Operations with Multi-stage Learningcited by 0
- 2026Auditable Graph-Guided Root Cause Analysis for Kubernetes Incidentscited by 0
- 2026Beyond Fault Localization: A Trajectory-Level Study of LLM Agents for Microservice Root Cause Analysiscited by 0
- 2026EviRCA: Decoupling Evidence Extraction from Reasoning for Microservice Root-Cause Analysiscited by 0
- 2026EviRCA: Decoupling Evidence Extraction from Reasoning for Microservice Root-Cause Analysis17 Sep 2026
- 2026Beyond Fault Localization: A Trajectory-Level Study of LLM Agents for Microservice Root Cause Analysis21 Aug 2026
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model14 Aug 2026
- 2026RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enough5 Aug 2026
- 2026Pooled Leaderboards Hide System-Specific Winners: A Reporting-Protocol Audit of Offline Root-Cause Analysis Benchmarks28 Jun 2026
- 2026Auditable Graph-Guided Root Cause Analysis for Kubernetes Incidents7 Jun 2026
- 2026CUJBench: Benchmarking LLM-Agent on Cross-Modal Failure Diagnosis from Browser to Backend25 Apr 2026
- 2026OpsLLM: Construction of Large Language Model for Software Operations with Multi-stage Learning6 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026Stalled, Biased, and Confused: Uncovering Reasoning Failures in LLMs for Cloud-Based Root Cause Analysis
- 2023RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Models
- 2026Cloud-OpsBench: A Reproducible Benchmark for Agentic Root Cause Analysis in Cloud Systems
- 2025Where LLM Agents Fail and How They can Learn From Failures
- 2026RIVA: Leveraging LLM Agents for Reliable Configuration Drift Detection
- 2024Building AI Agents for Autonomous Clouds: Challenges and Design Principles
- 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems
- 2023PACE-LM: Prompting and Augmentation for Calibrated Confidence Estimation with GPT-4 in Cloud Incident Root Cause Analysis