mlsanityData to 7 Oct 2026

On the Reliability of Large Language Models for Causal Discovery

arXiv 2407.19638 ↗First posted 29 Jul 2024cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

This study investigates the efficacy of Large Language Models (LLMs) in causal discovery. Using newly available open-source LLMs, OLMo and BLOOM, which provide access to their pre-training corpora, we investigate how LLMs address causal discovery through three research questions. We examine: (i) the impact of memorization for accurate causal relation prediction, (ii) the influence of incorrect causal relations in pre-training data, and (iii) the contextual nuances that influence LLMs' understanding of causal relations. Our findings indicate that while LLMs are effective in recognizing causal relations that occur frequently in pre-training data, their ability to generalize to new or rare causal relations is limited. Moreover, the presence of incorrect causal relations significantly undermines the confidence of LLMs in corresponding correct causal relations, and the contextual information critically affects the outcomes of LLMs to discern causal connections between random variables.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 18 of its 60 references that are in the map. First posted on arXiv 29 Jul 2024.

Builds on

18 of 60 references.

References · 18

  1. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  2. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  3. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  4. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  5. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
  6. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 348
  7. 2013Learning Bayesian Networks: The Combination of Knowledge and Statistical Datacited by 185
  8. 2021Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Littlecited by 176
  9. 2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
  10. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 146
  11. 2023What's In My Big Data?cited by 129
  12. 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 121
6 more
  1. 2023Can large language models build causal graphs?cited by 72
  2. 2021Frequency Effects on Syntactic Rule Learning in Transformerscited by 59
  3. 2023CRAB: Assessing the Strength of Causal Relationships Between Real-world Eventscited by 27
  4. 2023Enhancing Causal Reasoning in Large Language Models: A Causal Attribution Model for Precision Fine-Tuningcited by 14
  5. 2023Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue Response Generation Models by Causal Discoverycited by 10
  6. 2024CausalScore: An Automatic Reference-Free Metric for Assessing Response Relevance in Open-Domain Dialogue Systemscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 9.

12 papers.

  1. 2026Caliper: Probing Lexical Anchors versus Causal Structure in LLMscited by 2
  2. 2025Two-way Evidence self-Alignment based Dual-Gated Reasoning Enhancementcited by 1
  3. 2026Consistency evaluation of benchmarks used for causal discoverycited by 1
  4. 2026LLM Explainability with Counterfactual Chains and Causal Graphscited by 1
  5. 2025Causal Reasoning Elicits Controllable 3D Scene Generationcited by 0
  6. 2025Encoder Fine-tuning with Stochastic Sampling Outperforms Open-weight GPT in Astronomy Knowledge Extractioncited by 0
  7. 2026CAMO: An Agentic Framework for Automated Causal Discovery from Micro Behaviors to Macro Emergence in LLM Agent Simulationscited by 0
  8. 2026A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracingcited by 0
  9. 2026KRCA: An Efficient Root Cause Analysis System in Hyper-scale Microservice Systems via Agentic AIcited by 0
  10. 2026Integrating Background Knowledge for Scalable Causal Discoverycited by 0
  11. 2026From Causal Plausibility to Causal Reliability: Evaluating LLMs as Calibrated Direct Causal-Edge Classifierscited by 0
  12. 2026ABSOL: Aggregated Bayesian Subsampling Orchestrated with LLMscited by 0