# LLM-initialized Differentiable Causal Discovery

arXiv: https://arxiv.org/abs/2410.21141 · HTML: https://mlsanity.com/paper/2410.21141/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 28 Oct 2024.

## Builds on (most cited first, 11 of 11)

- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [Differentiable Causal Discovery from Interventional Data](https://mlsanity.com/paper/2007.01754/) (2020)
- [DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization](https://mlsanity.com/paper/2209.08037/) (2022)
- [CLadder: Assessing Causal Reasoning in Language Models](https://mlsanity.com/paper/2312.04350/) (2023)
- [Causal Discovery with Language Models as Imperfect Experts](https://mlsanity.com/paper/2307.02390/) (2023)
- [Integrating Large Language Model for Improved Causal Discovery](https://mlsanity.com/paper/2306.16902/) (2023)
- [Efficient Causal Graph Discovery Using Large Language Models](https://mlsanity.com/paper/2402.01207/) (2024)
- [Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference](https://mlsanity.com/paper/2310.15117/) (2023)
- [Large Language Models are Effective Priors for Causal Graph Discovery](https://mlsanity.com/paper/2405.13551/) (2024)
- [Dagma-DCE: Interpretable, Non-Parametric Differentiable Causal Discovery](https://mlsanity.com/paper/2401.02930/) (2024)

## Built on by (most cited first, 6 of 6)

- [LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discovery](https://mlsanity.com/paper/2506.00844/) (2025)
- [Revealing Multimodal Causality with Large Language Models](https://mlsanity.com/paper/2509.17784/) (2025)
- [DMCD: Semantic-Statistical Framework for Causal Discovery](https://mlsanity.com/paper/2602.20333/) (2026)
- [A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing](https://mlsanity.com/paper/2606.05330/) (2026)
- [From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning](https://mlsanity.com/paper/2606.11745/) (2026)
- [Integrating Background Knowledge for Scalable Causal Discovery](https://mlsanity.com/paper/2607.10456/) (2026)

## Abstract (arXiv)

The discovery of causal relationships between random variables is an important yet challenging problem that has applications across many scientific domains. Differentiable causal discovery (DCD) methods are effective in uncovering causal relationships from observational data; however, these approaches often suffer from limited interpretability and face challenges in incorporating domain-specific prior knowledge. In contrast, Large Language Models (LLMs)-based causal discovery approaches have recently been shown capable of providing useful priors for causal discovery but struggle with formal causal reasoning. In this paper, we propose LLM-DCD, which uses an LLM to initialize the optimization of the maximum likelihood objective function of DCD approaches, thereby incorporating strong priors into the discovery method. To achieve this initialization, we design our objective function to depend on an explicitly defined adjacency matrix of the causal graph as its only variational parameter. Directly optimizing the explicitly defined adjacency matrix provides a more interpretable approach to causal discovery. Additionally, we demonstrate higher accuracy on key benchmarking datasets of our approach compared to state-of-the-art alternatives, and provide empirical evidence that the quality of the initialization directly impacts the quality of the final output of our DCD approach. LLM-DCD opens up new opportunities for traditional causal discovery methods like DCD to benefit from future improvements in the causal reasoning capabilities of LLMs.