# Understanding Causality with Large Language Models: Feasibility and Opportunities

arXiv: https://arxiv.org/abs/2304.05524 · HTML: https://mlsanity.com/paper/2304.05524/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 11 Apr 2023.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Complexity-Based Prompting for Multi-Step Reasoning](https://mlsanity.com/paper/2210.00720/) (2022)
- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [Causal Structure Learning](https://mlsanity.com/paper/1706.09141/) (2017)
- [Shaking the foundations: delusions in sequence models for interaction and control](https://mlsanity.com/paper/2110.10819/) (2021)
- [Toward Self-learning End-to-End Task-Oriented Dialog Systems](https://mlsanity.com/paper/2201.06849/) (2022)

## Built on by (most cited first, 30 of 53)

- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [CLadder: Assessing Causal Reasoning in Language Models](https://mlsanity.com/paper/2312.04350/) (2023)
- [DPLM-2: A Multimodal Diffusion Protein Language Model](https://mlsanity.com/paper/2410.13782/) (2024)
- [Integrating Large Language Model for Improved Causal Discovery](https://mlsanity.com/paper/2306.16902/) (2023)
- [Large Language Models are Temporal and Causal Reasoners for Video Question Answering](https://mlsanity.com/paper/2310.15747/) (2023)
- [Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning](https://mlsanity.com/paper/2308.12219/) (2023)
- [Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?](https://mlsanity.com/paper/2506.21215/) (2025)
- [The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis](https://mlsanity.com/paper/2311.00237/) (2023)
- [The Essential Role of Causality in Foundation World Models for Embodied AI](https://mlsanity.com/paper/2402.06665/) (2024)
- [CRAB: Assessing the Strength of Causal Relationships Between Real-world Events](https://mlsanity.com/paper/2311.04284/) (2023)
- [Causality for Large Language Models](https://mlsanity.com/paper/2410.15319/) (2024)
- [Cause and Effect: Can Large Language Models Truly Understand Causality?](https://mlsanity.com/paper/2402.18139/) (2024)
- [Prompting Fairness: Integrating Causality to Debias Large Language Models](https://mlsanity.com/paper/2403.08743/) (2024)
- [CounterBench: Evaluating and Improving Counterfactual Reasoning in Large Language Models](https://mlsanity.com/paper/2502.11008/) (2025)
- [Causal Evaluation of Language Models](https://mlsanity.com/paper/2405.00622/) (2024)
- [Towards Causal Foundation Model: on Duality between Causal Inference and Attention](https://mlsanity.com/paper/2310.00809/) (2023)
- [An Interdisciplinary Outlook on Large Language Models for Scientific Research](https://mlsanity.com/paper/2311.04929/) (2023)
- [Zero-shot Causal Graph Extrapolation from Text via LLMs](https://mlsanity.com/paper/2312.14670/) (2023)
- [GPT-Based Models Meet Simulation: How to Efficiently Use Large-Scale Pre-Trained Language Models Across Simulation Tasks](https://mlsanity.com/paper/2306.13679/) (2023)
- [A Critical Review of Causal Reasoning Benchmarks for Large Language Models](https://mlsanity.com/paper/2407.08029/) (2024)
- [Reasoning Elicitation in Language Models via Counterfactual Feedback](https://mlsanity.com/paper/2410.03767/) (2024)
- [Limits of Transformer Language Models on Learning to Compose Algorithms](https://mlsanity.com/paper/2402.05785/) (2024)
- [Cloud Atlas: Efficient Fault Localization for Cloud Systems using Language Models and Causal Insight](https://mlsanity.com/paper/2407.08694/) (2024)
- [Failure Modes of LLMs for Causal Reasoning on Narratives](https://mlsanity.com/paper/2410.23884/) (2024)
- [Can Large Language Models Help Experimental Design for Causal Discovery?](https://mlsanity.com/paper/2503.01139/) (2025)
- [Discovering and Reasoning of Causality in the Hidden World with Large Language Models](https://mlsanity.com/paper/2402.03941/) (2024)
- [CELLO: Causal Evaluation of Large Vision-Language Models](https://mlsanity.com/paper/2406.19131/) (2024)
- [LLM4Causal: Democratized Causal Tools for Everyone via Large Language Model](https://mlsanity.com/paper/2312.17122/) (2023)
- [CHECKWHY: Causal Fact Verification via Argument Structure](https://mlsanity.com/paper/2408.10918/) (2024)
- [Applying Large Language Models for Causal Structure Learning in Non Small Cell Lung Cancer](https://mlsanity.com/paper/2311.07191/) (2023)

## Abstract (arXiv)

We assess the ability of large language models (LLMs) to answer causal questions by analyzing their strengths and weaknesses against three types of causal question. We believe that current LLMs can answer causal questions with existing causal knowledge as combined domain experts. However, they are not yet able to provide satisfactory answers for discovering new knowledge or for high-stakes decision-making tasks with high precision. We discuss possible future directions and opportunities, such as enabling explicit and implicit causal modules as well as deep causal-aware LLMs. These will not only enable LLMs to answer many different types of causal questions for greater impact but also enable LLMs to be more trustworthy and efficient in general.