# Large Language Models are Effective Priors for Causal Graph Discovery

arXiv: https://arxiv.org/abs/2405.13551 · HTML: https://mlsanity.com/paper/2405.13551/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 37 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 18 of 18)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [On the Planning Abilities of Large Language Models : A Critical Investigation](https://mlsanity.com/paper/2305.15771/) (2023)
- [Thinking Fast and Slow with Deep Learning and Tree Search](https://mlsanity.com/paper/1705.08439/) (2017)
- [Causal Inference and Causal Explanation with Background Knowledge](https://mlsanity.com/paper/1302.4972/) (2013)
- [Can Large Language Models Infer Causation from Correlation?](https://mlsanity.com/paper/2306.05836/) (2023)
- [Can large language models build causal graphs?](https://mlsanity.com/paper/2303.05279/) (2023)
- [Causal Discovery with Language Models as Imperfect Experts](https://mlsanity.com/paper/2307.02390/) (2023)
- [LMPriors: Pre-Trained Language Models as Task-Specific Priors](https://mlsanity.com/paper/2210.12530/) (2022)
- [Integrating Large Language Model for Improved Causal Discovery](https://mlsanity.com/paper/2306.16902/) (2023)
- [Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference](https://mlsanity.com/paper/2310.15117/) (2023)
- [Can Foundation Models Talk Causality?](https://mlsanity.com/paper/2206.10591/) (2022)
- [Causal BERT : Language models for causality detection between events expressed in text](https://mlsanity.com/paper/2012.05453/) (2020)
- [Typing assumptions improve identification in causal discovery](https://mlsanity.com/paper/2107.10703/) (2021)
- [Planning Spatial Networks with Monte Carlo Tree Search](https://mlsanity.com/paper/2106.06768/) (2021)
- [Tree Search in DAG Space with Model-based Reinforcement Learning for Causal Discovery](https://mlsanity.com/paper/2310.13576/) (2023)

## Built on by (most cited first, 24 of 24)

- [Multi-Agent Causal Discovery Using Large Language Models](https://mlsanity.com/paper/2407.15073/) (2024)
- [Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches](https://mlsanity.com/paper/2501.03151/) (2025)
- [A Survey on Enhancing Causal Reasoning Ability of Large Language Models](https://mlsanity.com/paper/2503.09326/) (2025)
- [LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discovery](https://mlsanity.com/paper/2506.00844/) (2025)
- [LLM-initialized Differentiable Causal Discovery](https://mlsanity.com/paper/2410.21141/) (2024)
- [CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientists](https://mlsanity.com/paper/2605.26029/) (2026)
- [Revealing Multimodal Causality with Large Language Models](https://mlsanity.com/paper/2509.17784/) (2025)
- [Score-informed Neural Operator for Enhancing Ordering-based Causal Discovery](https://mlsanity.com/paper/2508.12650/) (2025)
- [Incorporating Expert Knowledge into Bayesian Causal Discovery of Mixtures of Directed Acyclic Graphs](https://mlsanity.com/paper/2510.06735/) (2025)
- [IV Co-Scientist: Multi-Agent LLM Framework for Causal Instrumental Variable Discovery](https://mlsanity.com/paper/2602.07943/) (2026)
- [DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems](https://mlsanity.com/paper/2609.04749/) (2026)
- [Can LLMs Assist Expert Elicitation for Probabilistic Causal Modeling?](https://mlsanity.com/paper/2504.10397/) (2025)
- [CARE: Turning LLMs Into Causal Reasoning Expert](https://mlsanity.com/paper/2511.16016/) (2025)
- [KG-SoftMAP: Soft Knowledge-Graph Priors for Bayesian Network Structure Learning from Sparse Discrete Data](https://mlsanity.com/paper/2606.10358/) (2026)
- [Step-by-Step Causality: Transparent Causal Discovery with Multi-Agent Tree-Query and Adversarial Confidence Estimation](https://mlsanity.com/paper/2601.10137/) (2026)
- [DMCD: Semantic-Statistical Framework for Causal Discovery](https://mlsanity.com/paper/2602.20333/) (2026)
- [From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning](https://mlsanity.com/paper/2606.11745/) (2026)
- [Structured Inference with Large Language Gibbs](https://mlsanity.com/paper/2606.19264/) (2026)
- [Causal Discovery in the Era of Agents](https://mlsanity.com/paper/2606.23608/) (2026)
- [DKCD: Domain Knowledge-Enhanced Causal Discovery from Unstructured Data](https://mlsanity.com/paper/2607.09348/) (2026)
- [EviDAG: Auditable Causal DAG Authoring with Biomedical Literature](https://mlsanity.com/paper/2607.21859/) (2026)
- [Guide, Not Bind: Why Defeasible Priors Fail in Augmented Lagrangian Causal Discovery](https://mlsanity.com/paper/2609.03442/) (2026)
- [ABSOL: Aggregated Bayesian Subsampling Orchestrated with LLMs](https://mlsanity.com/paper/2609.15007/) (2026)
- [Coupling Noisy Pairwise Knowledge to the DAG Posterior for Causal Discovery](https://mlsanity.com/paper/2610.04559/) (2026)

## Abstract (arXiv)

Causal structure discovery from observations can be improved by integrating background knowledge provided by an expert to reduce the hypothesis space. Recently, Large Language Models (LLMs) have begun to be considered as sources of prior information given the low cost of querying them relative to a human expert. In this work, firstly, we propose a set of metrics for assessing LLM judgments for causal graph discovery independently of the downstream algorithm. Secondly, we systematically study a set of prompting designs that allows the model to specify priors about the structure of the causal graph. Finally, we present a general methodology for the integration of LLM priors in graph discovery algorithms, finding that they help improve performance on common-sense benchmarks and especially when used for assessing edge directionality. Our work highlights the potential as well as the shortcomings of the use of LLMs in this problem space.