# Integrating Large Language Models in Causal Discovery: A Statistical Causal Approach

arXiv: https://arxiv.org/abs/2402.01454 · HTML: https://mlsanity.com/paper/2402.01454/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 59 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 20 of 20)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [Generated Knowledge Prompting for Commonsense Reasoning](https://mlsanity.com/paper/2110.08387/) (2021)
- [Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game](https://mlsanity.com/paper/2102.13647/) (2021)
- [Causal Parrots: Large Language Models May Talk Causality But Are Not Causal](https://mlsanity.com/paper/2308.13067/) (2023)
- [Can Large Language Models Infer Causation from Correlation?](https://mlsanity.com/paper/2306.05836/) (2023)
- [Score matching enables causal discovery of nonlinear additive noise models](https://mlsanity.com/paper/2203.04413/) (2022)
- [Causal-learn: Causal Discovery in Python](https://mlsanity.com/paper/2307.16405/) (2023)
- [Causal Discovery with Language Models as Imperfect Experts](https://mlsanity.com/paper/2307.02390/) (2023)
- [Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference](https://mlsanity.com/paper/2310.15117/) (2023)
- [A Survey on Causal Discovery Methods for I.I.D. and Time Series Data](https://mlsanity.com/paper/2303.15027/) (2023)
- [Evaluation Methods and Measures for Causal Learning Algorithms](https://mlsanity.com/paper/2202.02896/) (2022)
- [Causal BERT : Language models for causality detection between events expressed in text](https://mlsanity.com/paper/2012.05453/) (2020)
- [Causal Structure Learning Supervised by Large Language Model](https://mlsanity.com/paper/2311.11689/) (2023)
- [Optimal transport for causal discovery](https://mlsanity.com/paper/2201.09366/) (2022)
- [LLM4Causal: Democratized Causal Tools for Everyone via Large Language Model](https://mlsanity.com/paper/2312.17122/) (2023)
- [Evaluation of Induced Expert Knowledge in Causal Structure Learning by NOTEARS](https://mlsanity.com/paper/2301.01817/) (2023)

## Built on by (most cited first, 30 of 32)

- [Causal Inference with Large Language Model: A Survey](https://mlsanity.com/paper/2409.09822/) (2024)
- [ALCM: Autonomous LLM-Augmented Causal Discovery Framework](https://mlsanity.com/paper/2405.01744/) (2024)
- [Multi-Agent Causal Discovery Using Large Language Models](https://mlsanity.com/paper/2407.15073/) (2024)
- [Causal Graph Discovery with Retrieval-Augmented Generation based Large Language Models](https://mlsanity.com/paper/2402.15301/) (2024)
- [Cloud Atlas: Efficient Fault Localization for Cloud Systems using Language Models and Causal Insight](https://mlsanity.com/paper/2407.08694/) (2024)
- [Exploring Multi-Modal Data with Tool-Augmented LLM Agents for Precise Causal Discovery](https://mlsanity.com/paper/2412.13667/) (2024)
- [LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discovery](https://mlsanity.com/paper/2506.00844/) (2025)
- [Mining Causality: AI-Assisted Search for Instrumental Variables](https://mlsanity.com/paper/2409.14202/) (2024)
- [Evaluating Interventional Reasoning Capabilities of Large Language Models](https://mlsanity.com/paper/2404.05545/) (2024)
- [Fairness-Driven LLM-based Causal Discovery with Active Learning and Dynamic Scoring](https://mlsanity.com/paper/2503.17569/) (2025)
- [Can LLMs Leverage Observational Data? Towards Data-Driven Causal Discovery with LLMs](https://mlsanity.com/paper/2504.10936/) (2025)
- [Causal Inference with Complex Treatments: A Survey](https://mlsanity.com/paper/2407.14022/) (2024)
- [Using GPT-4 to guide causal machine learning](https://mlsanity.com/paper/2407.18607/) (2024)
- [Uncovering Bias Paths with LLM-guided Causal Discovery: An Active Learning and Dynamic Scoring Approach](https://mlsanity.com/paper/2506.12227/) (2025)
- [Revealing Multimodal Causality with Large Language Models](https://mlsanity.com/paper/2509.17784/) (2025)
- [CauScientist: Teaching LLMs to Respect Data for Causal Discovery](https://mlsanity.com/paper/2601.13614/) (2026)
- [Can LLMs Assist Expert Elicitation for Probabilistic Causal Modeling?](https://mlsanity.com/paper/2504.10397/) (2025)
- [Improving constraint-based discovery with robust propagation and reliable LLM priors](https://mlsanity.com/paper/2509.23570/) (2025)
- [Consistency evaluation of benchmarks used for causal discovery](https://mlsanity.com/paper/2606.01789/) (2026)
- [CauTion: Knowing When to Trust LLMs for Ensemble Causal Discovery](https://mlsanity.com/paper/2606.03602/) (2026)
- [Sequential Causal Discovery with Noisy Language Model Priors](https://mlsanity.com/paper/2506.16234/) (2025)
- [Group Interventions on Deep Networks for Causal Discovery in Subsystems](https://mlsanity.com/paper/2510.23906/) (2025)
- [Linear Causal Discovery with Interventional Constraints](https://mlsanity.com/paper/2510.26342/) (2025)
- [DMCD: Semantic-Statistical Framework for Causal Discovery](https://mlsanity.com/paper/2602.20333/) (2026)
- [CAMO: An Agentic Framework for Automated Causal Discovery from Micro Behaviors to Macro Emergence in LLM Agent Simulations](https://mlsanity.com/paper/2604.14691/) (2026)
- [Eliciting associations between clinical variables from LLMs via comparison questions across populations](https://mlsanity.com/paper/2605.06335/) (2026)
- [Causal Discovery in the Era of Agents](https://mlsanity.com/paper/2606.23608/) (2026)
- [CausalSteward: An Agentic Divide-Conquer-Combine Copilot for Causal Discovery](https://mlsanity.com/paper/2607.01936/) (2026)
- [Integrating Background Knowledge for Scalable Causal Discovery](https://mlsanity.com/paper/2607.10456/) (2026)
- [ABSOL: Aggregated Bayesian Subsampling Orchestrated with LLMs](https://mlsanity.com/paper/2609.15007/) (2026)

## Abstract (arXiv)

In practical statistical causal discovery (SCD), embedding domain expert knowledge as constraints into the algorithm is important for reasonable causal models reflecting the broad knowledge of domain experts, despite the challenges in the systematic acquisition of background knowledge. To overcome these challenges, this paper proposes a novel method for causal inference, in which SCD and knowledge-based causal inference (KBCI) with a large language model (LLM) are synthesized through ``statistical causal prompting (SCP)'' for LLMs and prior knowledge augmentation for SCD. The experiments in this work have revealed that the results of LLM-KBCI and SCD augmented with LLM-KBCI approach the ground truths, more than the SCD result without prior knowledge. These experiments have also revealed that the SCD result can be further improved if the LLM undergoes SCP. Furthermore, with an unpublished real-world dataset, we have demonstrated that the background knowledge provided by the LLM can improve the SCD on this dataset, even if this dataset has never been included in the training data of the LLM. For future practical application of this proposed method across important domains such as healthcare, we also thoroughly discuss the limitations, risks of critical errors, expected improvement of techniques around LLMs, and realistic integration of expert checks of the results into this automatic process, with SCP simulations under various conditions both in successful and failure scenarios. The careful and appropriate application of the proposed approach in this work, with improvement and customization for each domain, can thus address challenges such as dataset biases and limitations, illustrating the potential of LLMs to improve data-driven causal inference across diverse scientific domains. The code used in this work is publicly available at: www.github.com/mas-takayama/LLM-and-SCD