# Why LLMs Fail at Causal Discovery and How Interventional Agents Escape

arXiv: https://arxiv.org/abs/2605.27567 · HTML: https://mlsanity.com/paper/2605.27567/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Causal reasoning in LLMs, one of 17 directions in LLM reasoning.
- Builds on 21 of its 26 references that are in the map.
- First posted on arXiv 26 May 2026.

## Placed in

- LLM reasoning / [Causal reasoning in LLMs](https://mlsanity.com/lr/causal-reasoning-in-llms/)
- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 21 of 21)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Causal Parrots: Large Language Models May Talk Causality But Are Not Causal](https://mlsanity.com/paper/2308.13067/) (2023)
- [Can Large Language Models Infer Causation from Correlation?](https://mlsanity.com/paper/2306.05836/) (2023)
- [Limitations of Lazy Training of Two-layers Neural Networks](https://mlsanity.com/paper/1906.08899/) (2019)
- [Understanding Causality with Large Language Models: Feasibility and Opportunities](https://mlsanity.com/paper/2304.05524/) (2023)
- [ABCD-Strategy: Budgeted Experimental Design for Targeted Causal Structure Discovery](https://mlsanity.com/paper/1902.10347/) (2019)
- [Learning Neural Causal Models with Active Interventions](https://mlsanity.com/paper/2109.02429/) (2021)
- [Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?](https://mlsanity.com/paper/2506.21215/) (2025)
- [The Essential Role of Causality in Foundation World Models for Embodied AI](https://mlsanity.com/paper/2402.06665/) (2024)
- [Multi-Agent Causal Discovery Using Large Language Models](https://mlsanity.com/paper/2407.15073/) (2024)
- [Causality for Large Language Models](https://mlsanity.com/paper/2410.15319/) (2024)
- [CausalGraph2LLM: Evaluating LLMs for Causal Queries](https://mlsanity.com/paper/2410.15939/) (2024)
- [Failure Modes of LLMs for Causal Reasoning on Narratives](https://mlsanity.com/paper/2410.23884/) (2024)
- [Can Large Language Models Help Experimental Design for Causal Discovery?](https://mlsanity.com/paper/2503.01139/) (2025)
- [LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discovery](https://mlsanity.com/paper/2506.00844/) (2025)
- [Prompting Strategies for Enabling Large Language Models to Infer Causation from Correlation](https://mlsanity.com/paper/2412.13952/) (2024)
- [On the Role of Transformer Feed-Forward Layers in Nonlinear In-Context Learning](https://mlsanity.com/paper/2501.18187/) (2025)
- [Provable In-Context Learning of Nonlinear Regression with Transformers](https://mlsanity.com/paper/2507.20443/) (2025)
- [Causal Reasoning in Pieces: Modular In-Context Learning for Causal Discovery](https://mlsanity.com/paper/2507.23488/) (2025)
- [Structured Thinking Matters: Improving LLMs Generalization in Causal Inference Tasks](https://mlsanity.com/paper/2505.18034/) (2025)
- [Leveraging Large Language Models for Causal Discovery: a Constraint-based, Argumentation-driven Approach](https://mlsanity.com/paper/2602.16481/) (2026)

## Built on by (most cited first, 1 of 1)

- [Capability-Gated Planning: Cost-to-Goal Discovery and the Limits of Myopic Experiment Selection](https://mlsanity.com/paper/2608.05085/) (2026)

## Abstract (arXiv)

Causal discovery is a cornerstone of scientific reasoning, yet whether large language models can perform it reliably remains an open question. Recent benchmarks show that even fine-tuned models plateau on simple causal graphs and degrade as complexity grows, but why they fail has not been established. We prove the failure is fundamental: supervised fine-tuning, direct preference optimization, and in-context learning all produce predictors that cannot distinguish between causal graphs generating similar observational data, and any attempt to do so requires the model's internal representations to grow unboundedly, violating the very conditions under which these methods work. We formalize this as a kernel obstruction theorem, establishing that the limitation is intrinsic to the learning paradigm, not any particular model or dataset. We propose Agentic Causal Bayesian Optimization (A-CBO), wherein a frozen language model serves as an interventional oracle answering targeted queries about intervention effects, while an external Bayesian loop concentrates beliefs over candidate graphs in logarithmically many rounds. Because the decision operates outside the space where the obstruction applies, A-CBO provably converges while the underlying model remains unchanged. On Corr2Cause, A-CBO matches fine-tuned baselines without any training. On Extended Corr2Cause, a new benchmark scaling to 24 variables with 18K test samples, A-CBO significantly outperforms both fine-tuning and preference optimization, with the advantage growing