# Integrating Large Language Model for Improved Causal Discovery

arXiv: https://arxiv.org/abs/2306.16902 · HTML: https://mlsanity.com/paper/2306.16902/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 75 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 28 of 28)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [D'ya like DAGs? A Survey on Structure Learning and Causal Discovery](https://mlsanity.com/paper/2103.02582/) (2021)
- [Learning Bayesian Networks: The Combination of Knowledge and Statistical Data](https://mlsanity.com/paper/1302.6815/) (2013)
- [SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks](https://mlsanity.com/paper/2305.17390/) (2023)
- [Can Large Language Models Infer Causation from Correlation?](https://mlsanity.com/paper/2306.05836/) (2023)
- [LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Models](https://mlsanity.com/paper/2305.13711/) (2023)
- [Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline](https://mlsanity.com/paper/2305.13144/) (2023)
- [Causal Structure Learning](https://mlsanity.com/paper/1706.09141/) (2017)
- [Learning Equivalence Classes of Bayesian Networks Structures](https://mlsanity.com/paper/1302.3566/) (2013)
- [Can large language models build causal graphs?](https://mlsanity.com/paper/2303.05279/) (2023)
- [LMPriors: Pre-Trained Language Models as Task-Specific Priors](https://mlsanity.com/paper/2210.12530/) (2022)
- [Understanding Causality with Large Language Models: Feasibility and Opportunities](https://mlsanity.com/paper/2304.05524/) (2023)
- [Causal-Discovery Performance of ChatGPT in the context of Neuropathic Pain Diagnosis](https://mlsanity.com/paper/2301.13819/) (2023)
- [The impact of prior knowledge on causal structure learning](https://mlsanity.com/paper/2102.00473/) (2021)
- [CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models](https://mlsanity.com/paper/2112.11941/) (2021)
- [Can Foundation Models Talk Causality?](https://mlsanity.com/paper/2206.10591/) (2022)
- [Theory Refinement on Bayesian Networks](https://mlsanity.com/paper/1303.5709/) (2013)
- [Learning Bayesian Networks: A Unification for Discrete and Gaussian Domains](https://mlsanity.com/paper/1302.4957/) (2013)
- [Bayesian Network Constraint-Based Structure Learning Algorithms: Parallel and Optimised Implementations in the bnlearn R Package](https://mlsanity.com/paper/1406.7648/) (2014)
- [Emerging Synergies in Causality and Deep Generative Models: A Survey](https://mlsanity.com/paper/2301.12351/) (2023)
- [Customizing General-Purpose Foundation Models for Medical Report Generation](https://mlsanity.com/paper/2306.05642/) (2023)
- [Directed Acyclic Graphs With Tears](https://mlsanity.com/paper/2302.02160/) (2023)
- [An Experimental Comparison of Hybrid Algorithms for Bayesian Network Structure Learning](https://mlsanity.com/paper/1505.05004/) (2015)

## Built on by (most cited first, 30 of 54)

- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMs](https://mlsanity.com/paper/2410.14052/) (2024)
- [Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference](https://mlsanity.com/paper/2310.15117/) (2023)
- [Causal Inference with Large Language Model: A Survey](https://mlsanity.com/paper/2409.09822/) (2024)
- [Belief Revision: The Adaptability of Large Language Models Reasoning](https://mlsanity.com/paper/2406.19764/) (2024)
- [ALCM: Autonomous LLM-Augmented Causal Discovery Framework](https://mlsanity.com/paper/2405.01744/) (2024)
- [Large Language Models and Causal Inference in Collaboration: A Survey](https://mlsanity.com/paper/2403.09606/) (2024)
- [TWIN-GPT: Digital Twins for Clinical Trials via Large Language Model](https://mlsanity.com/paper/2404.01273/) (2024)
- [Large Language Models are Effective Priors for Causal Graph Discovery](https://mlsanity.com/paper/2405.13551/) (2024)
- [Causal Structure Learning Supervised by Large Language Model](https://mlsanity.com/paper/2311.11689/) (2023)
- [Causal Evaluation of Language Models](https://mlsanity.com/paper/2405.00622/) (2024)
- [End-To-End Causal Effect Estimation from Unstructured Natural Language Data](https://mlsanity.com/paper/2407.07018/) (2024)
- [Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches](https://mlsanity.com/paper/2501.03151/) (2025)
- [Towards Causal Foundation Model: on Duality between Causal Inference and Attention](https://mlsanity.com/paper/2310.00809/) (2023)
- [Causal Graph Discovery with Retrieval-Augmented Generation based Large Language Models](https://mlsanity.com/paper/2402.15301/) (2024)
- [Large Language Models for Constrained-Based Causal Discovery](https://mlsanity.com/paper/2406.07378/) (2024)
- [RealTCD: Temporal Causal Discovery from Interventional Data with Large Language Model](https://mlsanity.com/paper/2404.14786/) (2024)
- [Teaching Transformers Causal Reasoning through Axiomatic Training](https://mlsanity.com/paper/2407.07612/) (2024)
- [CausalGraph2LLM: Evaluating LLMs for Causal Queries](https://mlsanity.com/paper/2410.15939/) (2024)
- [Discovering and Reasoning of Causality in the Hidden World with Large Language Models](https://mlsanity.com/paper/2402.03941/) (2024)
- [Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Code](https://mlsanity.com/paper/2510.01539/) (2025)
- [IFAR: Multi-Perspective and Multi-Level Causal Discovery with LLMs](https://mlsanity.com/paper/2409.05559/) (2024)
- [A Survey on Enhancing Causal Reasoning Ability of Large Language Models](https://mlsanity.com/paper/2503.09326/) (2025)
- [LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discovery](https://mlsanity.com/paper/2506.00844/) (2025)
- [LLM4GRN: Discovering Causal Gene Regulatory Networks with LLMs -- Evaluation through Synthetic Data Generation](https://mlsanity.com/paper/2410.15828/) (2024)
- [Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models](https://mlsanity.com/paper/2608.09696/) (2026)
- [Evaluating Interventional Reasoning Capabilities of Large Language Models](https://mlsanity.com/paper/2404.05545/) (2024)
- [LLM-initialized Differentiable Causal Discovery](https://mlsanity.com/paper/2410.21141/) (2024)
- [Can LLMs Leverage Observational Data? Towards Data-Driven Causal Discovery with LLMs](https://mlsanity.com/paper/2504.10936/) (2025)
- [Bayesian Network Structure Discovery Using Large Language Models](https://mlsanity.com/paper/2511.00574/) (2025)

## Abstract (arXiv)

Recovering the structure of causal graphical models from observational data is an essential yet challenging task for causal discovery in scientific scenarios. Domain-specific causal discovery usually relies on expert validation or prior analysis to improve the reliability of recovered causality, which is yet limited by the scarcity of expert resources. Recently, Large Language Models (LLM) have been used for causal analysis across various domain-specific scenarios, suggesting its potential as autonomous expert roles in guiding data-based structure learning. However, integrating LLMs into causal discovery faces challenges due to inaccuracies in LLM-based reasoning on revealing the actual causal structure. To address this challenge, we propose an error-tolerant LLM-driven causal discovery framework. The error-tolerant mechanism is designed three-fold with sufficient consideration on potential inaccuracies. In the LLM-based reasoning process, an accuracy-oriented prompting strategy restricts causal analysis to a reliable range. Next, a knowledge-to-structure transition aligns LLM-derived causal statements with structural causal interactions. In the structure learning process, the goodness-of-fit to data and adherence to LLM-derived priors are balanced to further address prior inaccuracies. Evaluation of eight real-world causal structures demonstrates the efficacy of our LLM-driven approach in improving data-based causal discovery, along with its robustness to inaccurate LLM-derived priors. Codes are available at https://github.com/tyMadara/LLM-CD.