# Reasoning Models Are Accurate but Unsound on Identification

arXiv: https://arxiv.org/abs/2610.03519 · HTML: https://mlsanity.com/paper/2610.03519/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Causal reasoning in LLMs, one of 17 directions in LLM reasoning.
- Builds on 31 of its 63 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Placed in

- LLM reasoning / [Causal reasoning in LLMs](https://mlsanity.com/lr/causal-reasoning-in-llms/)

## Builds on (most cited first, 30 of 31)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [Causal Parrots: Large Language Models May Talk Causality But Are Not Causal](https://mlsanity.com/paper/2308.13067/) (2023)
- [AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions](https://mlsanity.com/paper/2506.09038/) (2025)
- [CLadder: Assessing Causal Reasoning in Language Models](https://mlsanity.com/paper/2312.04350/) (2023)
- [Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations](https://mlsanity.com/paper/2302.02187/) (2023)
- [Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Data](https://mlsanity.com/paper/2402.17644/) (2024)
- [CogBench: a large language model walks into a psychology lab](https://mlsanity.com/paper/2402.18225/) (2024)
- [Can LLM Graph Reasoning Generalize beyond Pattern Memorization?](https://mlsanity.com/paper/2406.15992/) (2024)
- [Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?](https://mlsanity.com/paper/2506.21215/) (2025)
- [Identifying Causal Effects with the R Package causaleffect](https://mlsanity.com/paper/1806.07161/) (2018)
- [A Critical Review of Causal Reasoning Benchmarks for Large Language Models](https://mlsanity.com/paper/2407.08029/) (2024)
- [Graph Neural Network Causal Explanation via Neural Causal Models](https://mlsanity.com/paper/2407.09378/) (2024)
- [LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discovery](https://mlsanity.com/paper/2506.00844/) (2025)
- [Enhancing Identification of Causal Effects by Pruning](https://mlsanity.com/paper/1806.07085/) (2018)
- [Simplifying Probabilistic Expressions in Causal Inference](https://mlsanity.com/paper/1806.07082/) (2018)
- [CausalReasoningBenchmark: A Real-World Benchmark for Disentangled Evaluation of Causal Identification and Estimation](https://mlsanity.com/paper/2602.20571/) (2026)
- [Uncovering Hidden Correctness in LLM Causal Reasoning via Symbolic Verification](https://mlsanity.com/paper/2601.21210/) (2026)
- [Measure-Theoretic Anti-Causal Representation Learning](https://mlsanity.com/paper/2510.18052/) (2025)
- [Experimental Design for Causal Effect Identification](https://mlsanity.com/paper/2205.02232/) (2022)
- [Fast Proxy Experiment Design for Causal Effect Identification](https://mlsanity.com/paper/2407.05330/) (2024)
- [NoisyCausal: A Benchmark for Evaluating Causal Reasoning Under Structured Noise](https://mlsanity.com/paper/2605.04313/) (2026)
- [CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness](https://mlsanity.com/paper/2607.18820/) (2026)
- [On causal inference with marked point process data](https://mlsanity.com/paper/2604.12977/) (2026)

## Abstract (arXiv)

A reasoning model asked whether a causal effect is recoverable from observational data can fail in two ways: it refuses an identifiable query or answers a nonidentifiable one. The latter is more consequential, as no observational data can validate the claimed formula. Measuring this failure requires queries that are provably non-identifiable, which prior evaluations lack, and grading that accepts correct formulas in any equivalent form, which string matching cannot provide. We build CERTID, a formal identification pipeline that addresses both limitations. CERTID uses the sound and complete causal identification algorithm ID to certify whether an effect is identifiable from a given graph and query, and verifies returned formulas against structural causal models whose interventional distributions are known exactly. CERTID further develops theoretical results to mitigate structural leakage, repair non-identifiable queries, and establish grading guarantees. We evaluate three frontier reasoning models (Gemini Flash, Gemini Pro, and GPT5.5) on 1,200 certified instances spanning 4 to 50 vertices. Accuracy proves a poor proxy for soundness: on identical instances, the false-claim rate on non-identifiable queries varies by seventeen-fold across models. We also find that models decide identifiability with 97-100% accuracy on graphs generated after the strongest model's training snapshot. Instances, the certification procedure, the verifier, and per-instance records are available at https://anonymous.4open.science/r/certid-D718.