mlsanityData to 7 Oct 2026

RCAEval: A Benchmark for Root Cause Analysis of Microservice Systems with Telemetry Data

arXiv 2412.17015 ↗First posted 22 Dec 2024cs.SEcited by 26

Not on the map. Why?

Abstract (arXiv)

Root cause analysis (RCA) for microservice systems has gained significant attention in recent years. However, there is still no standard benchmark that includes large-scale datasets and supports comprehensive evaluation environments. In this paper, we introduce RCAEval, an open-source benchmark that provides datasets and an evaluation environment for RCA in microservice systems. First, we introduce three comprehensive datasets comprising 735 failure cases collected from three microservice systems, covering various fault types observed in real-world failures. Second, we present a comprehensive evaluation framework that includes fifteen reproducible baselines covering a wide range of RCA approaches, with the ability to evaluate both coarse-grained and fine-grained RCA. We hope that this ready-to-use benchmark will enable researchers and practitioners to conduct extensive analysis and pave the way for robust new solutions for RCA of microservice systems.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Dec 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 22.

26 papers.

Sort:

By area:

  1. 2026Cloud-OpsBench: A Reproducible Benchmark for Agentic Root Cause Analysis in Cloud Systemscited by 13
  2. 2026Why Do AI Agents Systematically Fail at Cloud Root Cause Analysis?cited by 8
  3. 2025GALA: Can Graph-Augmented Large Language Model Agentic Workflows Elevate Root Cause Analysis?cited by 5
  4. 2026OpenRCA 2.0: From Outcome Labels to Causal Process Supervisioncited by 5
  5. 2026Graph-Free Root Cause Analysiscited by 3
  6. 2025TN-AutoRCA: Benchmark Construction and Agentic Framework for Self-Improving Alarm-Based Root Cause Analysis in Telecommunication Networkscited by 2
  7. 2026ORCA-bench: How Ready Are Language Model Agents for Oncall?cited by 2
  8. 2025OpenDerisk: An Industrial Framework for AI-Driven SRE, with Design, Implementation, and Case Studiescited by 1
  9. 2026Bridging Temporal and Textual Modalities: A Multimodal Framework for Automated Cloud Failure Root Cause Analysiscited by 1
  10. 2026CUJBench: Benchmarking LLM-Agent on Cross-Modal Failure Diagnosis from Browser to Backendcited by 1
  11. 2026EventADL: Open-Box Anomaly Detection and Localization Framework for Events in Cloud-Based Service Systemscited by 1
  12. 2026TopoEvo: A Topology-Aware Self-Evolving Multi-Agent Framework for Root Cause Analysis in Microservicescited by 1
14 more
  1. 2026Pooled Leaderboards Hide System-Specific Winners: A Reporting-Protocol Audit of Offline Root-Cause Analysis Benchmarkscited by 1
  2. 2026A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosiscited by 1
  3. 2026TelemetrySuffBench: Is Agent Telemetry Sufficient for Failure-Origin Diagnosis?cited by 1
  4. 2025Autonomic Microservice Management via Agentic AI and MAPE-K Integrationcited by 0
  5. 2026STLGT: A Scalable Trace-Based Linear Graph Transformer for Tail Latency Prediction in Microservicescited by 0
  6. 2026OpsLLM: Construction of Large Language Model for Software Operations with Multi-stage Learningcited by 0
  7. 2026StableRCA: Robust Graph-Agnostic Mechanism-Level Root Cause Analysiscited by 0
  8. 2026How Far Can Root Cause Analysis Go on Real-World Telemetry Data?cited by 0
  9. 2026From General Agents to RCA Experts: A Self-Evolving Harness for Root Cause Analysiscited by 0
  10. 2026Does Graph Structure Earn Its Place in Microservice Root-Cause Analysis? A Controlled Study on RCAEval, and What the Benchmark Was Really Measuringcited by 0
  11. 2026Stochastic Semantic Evidence Graphs: Uncertainty Propagation and Governance for Agentic AIcited by 0
  12. 2026SRE-Marathon: A Continuous, Change-Driven Benchmark for Autonomous Site Reliability Agentscited by 0
  13. 2026Where Root Cause Analysis Fails: A Retrieval-Reranking Decompositioncited by 0
  14. 2026Beyond Conditional Independence: Root Cause Analysis with Deep Causal Modelscited by 0