mlsanityData to 14 Oct 2026

CausalEval: Towards Better Causal Reasoning in Language Models

arXiv 2410.16676 ↗First posted 22 Oct 2024cs.AI cs.CLcited by 13

On the map: LLM reasoning → Causal reasoning in LLMs

Abstract (arXiv)

Causal reasoning (CR) is a crucial aspect of intelligence, essential for problem-solving, decision-making, and understanding the world. While language models (LMs) can generate rationales for their outputs, their ability to reliably perform causal reasoning remains uncertain, often falling short in tasks requiring a deep understanding of causality. In this paper, we introduce CausalEval, a comprehensive review of research aimed at enhancing LMs for causal reasoning, coupled with an empirical evaluation of current models and methods. We categorize existing methods based on the role of LMs: either as reasoning engines or as helpers providing knowledge or data to traditional CR methods, followed by a detailed discussion of methodologies in each category. We then assess the performance of current LMs and various enhancement methods on a range of causal reasoning tasks, providing key findings and in-depth analysis. Finally, we present insights from current studies and highlight promising directions for future research. We aim for this work to serve as a comprehensive resource, fostering further advancements in causal reasoning with LMs.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Causal reasoning in LLMs, one of 17 directions in LLM reasoning. Builds on 61 of its 126 references that are in the map. First posted on arXiv 22 Oct 2024.

Builds on

61 of 126 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024C2P: Featuring Large Language Models with Causal Reasoning25 Jul
  2. 2024A Critical Review of Causal Reasoning Benchmarks for Large Language Models10 Jul
  3. 2024Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Data27 Feb
  4. 2023Do large language models and humans have similar behaviors in causal inference with script knowledge?13 Nov
  5. 2023Towards CausalGPT: A Multi-Agent Approach for Faithful Knowledge Reasoning via Promoting Causal Consistency in LLMs23 Aug
  6. 2023COLA: Contextualized Commonsense Causal Reasoning from the Causal Inference Perspective9 May
  7. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causality28 Apr
  8. 2023Causal Reasoning of Entities and Events in Procedural Texts26 Jan
  9. 2022e-CARE: a New Dataset for Exploring Explainable Causal Reasoning12 May

Other references · 52

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  9. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  10. 2022Large Language Models Can Self-Improvecited by 664
  11. 2021True Few-Shot Learning with Language Modelscited by 344
  12. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 334
38 more
  1. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  2. 2019Abductive Commonsense Reasoningcited by 301
  3. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 285
  4. 2021Internet-Augmented Dialogue Generationcited by 217
  5. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  6. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  7. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 172
  8. 2019Causal Discovery from Heterogeneous/Nonstationary Data with Independent Changescited by 166
  9. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 150
  10. 2023Can Large Language Models Infer Causation from Correlation?cited by 146
  11. 2019Causal Discovery with Reinforcement Learningcited by 145
  12. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 143
  13. 2022Faithful Reasoning Using Large Language Modelscited by 130
  14. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  15. 2019Counterfactual Story Reasoning and Generationcited by 88
  16. 2020GLUCOSE: GeneraLized and COntextualized Story Explanationscited by 86
  17. 2019Joint Reasoning for Temporal and Causal Relationscited by 62
  18. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  19. 2023ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Modelscited by 43
  20. 2023TRAM: Benchmarking Temporal Reasoning for Large Language Modelscited by 40
  21. 2019SocialIQA: Commonsense Reasoning about Social Interactionscited by 37
  22. 2024How Likely Do LLMs with CoT Mimic Human Reasoning?cited by 34
  23. 2023CRAB: Assessing the Strength of Causal Relationships Between Real-world Eventscited by 28
  24. 2024Large Language Models and Causal Inference in Collaboration: A Surveycited by 26
  25. 2024Large Language Models for Causal Discovery: Current Landscape and Future Directionscited by 25
  26. 2023Causal Structure Learning Supervised by Large Language Modelcited by 21
  27. 2023Causal Deep Learningcited by 20
  28. 2022CPL: Counterfactual Prompt Learning for Vision and Language Modelscited by 18
  29. 2018FASK with Interventional Knowledge Recovers Edges from the Sachs Modelcited by 14
  30. 2023Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inferencecited by 13
  31. 2024RealTCD: Temporal Causal Discovery from Interventional Data with Large Language Modelcited by 13
  32. 2021CausalBERT: Injecting Causal Knowledge Into Pre-trained Models with Minimal Supervisioncited by 12
  33. 2021Improving Commonsense Causal Reasoning by Adversarial Training and Data Augmentationcited by 9
  34. 2023LLM4Causal: Democratized Causal Tools for Everyone via Large Language Modelcited by 9
  35. 2023Extracting Self-Consistent Causal Insights from Users Feedback with LLMs and In-context Learningcited by 5
  36. 2022The interventional Bayesian Gaussian equivalent score for Bayesian causal inference with unknown soft interventionscited by 4
  37. 2023Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofscited by 2
  38. 2023Evidence-based Interpretable Open-domain Fact-checking with Large Language Modelscited by 2

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 4.

13 papers.

Sort:

By area:

  1. 2025Mitigating Hallucinations in Large Language Models via Causal Reasoningcited by 13
  2. 2025LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discoverycited by 8
  3. 2025Causal Consistency Regularization: Training Verifiably Sensitive Reasoning in Large Language Modelscited by 8
  4. 2025Probing the Preferences of a Language Model: Integrating Verbal and Behavioral Tests of AI Welfarecited by 7
  5. 2025ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Modelscited by 3
  6. 2025A Multi-Level Benchmark for Causal Language Understanding in Social Media Discoursecited by 3
  7. 2025Causally-Enhanced Reinforcement Policy Optimizationcited by 2
  8. 2026Epistemic Regret Minimization: Label-Free Causal Critique Beyond Outcome Rewardcited by 2
  9. 2025CRAwDAD: Causal Reasoning Augmentation with Dual-Agent Debatecited by 1
  10. 2026ClimateCause: Complex and Implicit Causal Structures in Climate Reportscited by 1
  11. 2025GRID: Graph-based Reasoning for Intervention and Discovery in Built Environmentscited by 0
  12. 2026Evaluating Bivariate Causal Statements Based on Mutual Compatibilitycited by 0
1 more
  1. 2026Contextual Causality with Large Language Models: A Surveycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Enhancing Causal Reasoning Ability of Large Language Models
  2. 2024Causal Evaluation of Language Models
  3. 2023CLadder: Assessing Causal Reasoning in Language Models
  4. 2025Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
  5. 2025Causality for Natural Language Processing
  6. 2023Enhancing Causal Reasoning in Large Language Models: A Causal Attribution Model for Precision Fine-Tuning
  7. 2024CARL-GT: Evaluating Causal Reasoning Capabilities of Large Language Models
  8. 2026CausalFlip: A Benchmark for LLM Causal Judgment Beyond Semantic Matching