# OpenRCA 2.0: From Outcome Labels to Causal Process Supervision

arXiv: https://arxiv.org/abs/2606.27154 · HTML: https://mlsanity.com/paper/2606.27154/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 25 Jun 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 14 of 14)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Solving math word problems with process- and outcome-based feedback](https://mlsanity.com/paper/2211.14275/) (2022)
- [A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges](https://mlsanity.com/paper/2412.11936/) (2024)
- [AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds](https://mlsanity.com/paper/2501.06706/) (2025)
- [VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data](https://mlsanity.com/paper/2502.06737/) (2025)
- [RCAEval: A Benchmark for Root Cause Analysis of Microservice Systems with Telemetry Data](https://mlsanity.com/paper/2412.17015/) (2024)
- [Constructing Large-Scale Real-World Benchmark Datasets for AIOps](https://mlsanity.com/paper/2208.03938/) (2022)
- [STRATUS: A Multi-agent System for Autonomous Reliability Engineering of Modern Clouds](https://mlsanity.com/paper/2506.02009/) (2025)
- [ThinkFL: Self-Refining Failure Localization for Microservice Systems via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2504.18776/) (2025)
- [Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain](https://mlsanity.com/paper/2603.02218/) (2026)
- [Dynamic and Generalizable Process Reward Modeling](https://mlsanity.com/paper/2507.17849/) (2025)
- [Rethinking the Evaluation of Microservice RCA with a Fault Propagation-Aware Benchmark](https://mlsanity.com/paper/2510.04711/) (2025)
- [DynaCausal: Dynamic Causality-Aware Root Cause Analysis for Distributed Microservices](https://mlsanity.com/paper/2510.22613/) (2025)

## Built on by (most cited first, 5 of 5)

- [Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures](https://mlsanity.com/paper/2607.28802/) (2026)
- [ORCA-bench: How Ready Are Language Model Agents for Oncall?](https://mlsanity.com/paper/2607.28545/) (2026)
- [Agent-Native Telemetry: Verifiable State-Delta Evidence for Autonomous Operations](https://mlsanity.com/paper/2608.16178/) (2026)
- [FinRCA-Bench: Benchmarking Evidence Retrieval and Reasoning for Financial AI Systems](https://mlsanity.com/paper/2608.18534/) (2026)
- [SRE-Marathon: A Continuous, Change-Driven Benchmark for Autonomous Site Reliability Agents](https://mlsanity.com/paper/2609.33023/) (2026)

## Abstract (arXiv)

Root cause analysis (RCA) poses a holistic test of LLM agentic capabilities, such as long-context understanding, multi-step reasoning, and tool use. However, existing datasets suffer from a fundamental gap: they label only the root cause, not the propagation path connecting it to the observed symptom, which largely simplifies the task to naive pattern matching. To support rigorous evaluation, we introduce PAVE, a step-wise labeling protocol that leverages known interventions from fault injection to reconstruct causal propagation paths. The mechanism is forward verification: reasoning from cause to effect rather than inferring backward from symptoms. Applying PAVE yields OpenRCA 2.0 (500 instances), the first cross-system RCA benchmark with step-wise causal annotations for LLM agents. Across 11 frontier LLMs, recovering the exact root-cause set succeeds in only 20.7% of cases on average. To locate where this difficulty lies, we relax the criterion and find what we call the ungrounded diagnosis: agents identify at least one correct root-cause service in 76.0% of cases, but ground that service in a verified causal propagation path to the observed symptom in only 61.5%. Outcome-only evaluation hides this failure mode; step-wise causal ground truth is the missing piece for trustworthy LLM-based RCA agents.