# C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning

arXiv: https://arxiv.org/abs/2603.05167 · HTML: https://mlsanity.com/paper/2603.05167/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 5 Mar 2026.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 14 of 14)

- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Solving math word problems with process- and outcome-based feedback](https://mlsanity.com/paper/2211.14275/) (2022)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [ProcessBench: Identifying Process Errors in Mathematical Reasoning](https://mlsanity.com/paper/2412.06559/) (2024)
- [Chain-of-Thought Reasoning In The Wild Is Not Always Faithful](https://mlsanity.com/paper/2503.08679/) (2025)
- [Thought Anchors: Which LLM Reasoning Steps Matter?](https://mlsanity.com/paper/2506.19143/) (2025)
- [PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models](https://mlsanity.com/paper/2501.03124/) (2025)
- [Are DeepSeek R1 And Other Reasoning Models More Faithful?](https://mlsanity.com/paper/2501.08156/) (2025)
- [FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2510.04040/) (2025)
- [Robustly Improving LLM Fairness in Realistic Settings via Interpretability](https://mlsanity.com/paper/2506.10922/) (2025)
- [A Pragmatic Way to Measure Chain-of-Thought Monitorability](https://mlsanity.com/paper/2510.23966/) (2025)

## Built on by (most cited first, 7 of 7)

- [Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2605.24286/) (2026)
- [Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth](https://mlsanity.com/paper/2605.25052/) (2026)
- [Strained Coherence: A Pre-Failure Signal in Coding Agent Execution Trajectories](https://mlsanity.com/paper/2606.07889/) (2026)
- [The Reasoning Trap: An Information-Theoretic Bound on Closed-System Multi-Step LLM Reasoning](https://mlsanity.com/paper/2605.01704/) (2026)
- [Externalized CPDAG Summaries Improve LLM Causal Deduction](https://mlsanity.com/paper/2609.31071/) (2026)
- [A Proposed Rubric for Evaluating Expressed Clinical Reasoning in Large Language Model Responses](https://mlsanity.com/paper/2609.37788/) (2026)
- [Hallucination Across the Reasoning Lifecycle: Interface Visibility, Causal Evidence, and Release Control in Large Reasoning Models](https://mlsanity.com/paper/2610.05472/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used as judges of chain-of-thought (CoT) reasoning, yet it remains unclear whether they can reliably assess process faithfulness rather than merely answer plausibility. We introduce C2-Faith, a benchmark built from PRM800K that explicitly decomposes faithfulness into two complementary dimensions: causality (whether each step logically follows from prior context) and coverage (whether essential intermediate inferences are present). Using controlled perturbations, we construct examples with known causal error positions by replacing a single step with a logically inconsistent variant, and with controlled coverage deletions at varying rates, enabling direct measurement against reference labels. We evaluate three frontier LLM judges across three tasks: binary causal detection, causal step localization, and coverage scoring. Our results reveal that judge reliability is highly task-dependent, with no single model dominating across settings. While models often detect that an error exists, they struggle to accurately localize it, indicating a substantial gap between detection and attribution. Moreover, all judges systematically overestimate reasoning completeness, assigning high coverage scores even when substantial portions of intermediate reasoning are missing. These findings expose fundamental limitations of LLM judges in process-level evaluation and highlight the need for more reliable and calibrated methods when using LLMs to assess reasoning quality.