Causal Reasoning in Pieces: Modular In-Context Learning for Causal Discovery
On the map: LLM reasoning → Causal reasoning in LLMs
Abstract (arXiv)
Causal inference remains a fundamental challenge for large language models. Recent advances in internal reasoning with large language models have sparked interest in whether state-of-the-art reasoning models can robustly perform causal discovery-a task where conventional models often suffer from severe overfitting and near-random performance under data perturbations. We study causal discovery on the Corr2Cause benchmark using the emergent OpenAI's o-series and DeepSeek-R model families and find that these reasoning-first architectures achieve significantly greater native gains than prior approaches. To capitalize on these strengths, we introduce a modular in-context pipeline inspired by the Tree-of-Thoughts and Chain-of-Thoughts methodologies, yielding nearly three-fold improvements over conventional baselines. We further probe the pipeline's impact by analyzing reasoning chain length, complexity, and conducting qualitative and quantitative comparisons between conventional and reasoning models. Our findings suggest that while advanced reasoning models represent a substantial leap forward, carefully structured in-context frameworks are essential to maximize their capabilities and offer a generalizable blueprint for causal discovery across diverse domains.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Causal reasoning in LLMs, one of 17 directions in LLM reasoning. Builds on 12 of its 16 references that are in the map. First posted on arXiv 31 Jul 2025.
Builds on
12 of 16 references.
Closest ancestors · 3
Same direction, newest first
- 2024Prompting Strategies for Enabling Large Language Models to Infer Causation from Correlation18 Dec
- 2024C2P: Featuring Large Language Models with Causal Reasoning25 Jul
- 2024Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Data27 Feb
Other references · 9
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
- 2013Causal Inference and Causal Explanation with Background Knowledgecited by 170
- 2023Can Large Language Models Infer Causation from Correlation?cited by 146
- 2024Leveraging LLM Reasoning Enhances Personalized Recommender Systemscited by 28
- 2024Large Language Models for Causal Discovery: Current Landscape and Future Directionscited by 25
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2023CLadder: Assessing Causal Reasoning in Language Models
- 2024A Critical Review of Causal Reasoning Benchmarks for Large Language Models
- 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causality
- 2023Enhancing Causal Reasoning in Large Language Models: A Causal Attribution Model for Precision Fine-Tuning
- 2026Leveraging Large Language Models for Causal Discovery: a Constraint-based, Argumentation-driven Approach
- 2024OCDB: Revisiting Causal Discovery with a Comprehensive Benchmark and Evaluation Framework
- 2026NoisyCausal: A Benchmark for Evaluating Causal Reasoning Under Structured Noise
- 2024Large Language Models for Constrained-Based Causal Discovery