mlsanityData to 14 Oct 2026

Can Large Language Models Infer Causation from Correlation?

arXiv 2306.05836 ↗First posted 9 Jun 2023cs.CL cs.AI cs.LGcited by 146

Not on the map. Why?

Cited by 55 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Causal inference is one of the hallmarks of human intelligence. While the field of CausalNLP has attracted much interest in the recent years, existing causal inference datasets in NLP primarily rely on discovering causality from empirical knowledge (e.g., commonsense knowledge). In this work, we propose the first benchmark dataset to test the pure causal inference skills of large language models (LLMs). Specifically, we formulate a novel task Corr2Cause, which takes a set of correlational statements and determines the causal relationship between the variables. We curate a large-scale dataset of more than 200K samples, on which we evaluate seventeen existing LLMs. Through our experiments, we identify a key shortcoming of LLMs in terms of their causal inference skills, and show that these models achieve almost close to random performance on the task. This shortcoming is somewhat mitigated when we try to re-purpose LLMs for this skill via finetuning, but we find that these models still fail to generalize -- they can only perform causal inference in in-distribution settings when variable names and textual expressions used in the queries are similar to those in the training set, but fail in out-of-distribution settings generated by perturbing these queries. Corr2Cause is a challenging task for LLMs, and would be helpful in guiding future research on improving LLMs' pure reasoning skills and generalizability. Our data is at https://huggingface.co/datasets/causalnlp/corr2cause. Our code is at https://github.com/causalNLP/corr2cause.

Key facts. Cited by 146 papers in the map (data through 14 Oct 2026). Builds on 21 of its 50 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

21 of 50 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  11. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  12. 2015A large annotated corpus for learning natural language inferencecited by 1,847
9 more
  1. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  2. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 353
  3. 2019Abductive Commonsense Reasoningcited by 301
  4. 2019Counterfactual Story Reasoning and Generationcited by 88
  5. 2020Pre-trained Summarization Distillationcited by 71
  6. 2020Reasoning about Goals, Steps, and Temporal Ordering with WikiHowcited by 66
  7. 2023Causal-Discovery Performance of ChatGPT in the context of Neuropathic Pain Diagnosiscited by 36
  8. 2022Logical Fallacy Detectioncited by 25
  9. 2023Multitask Instruction-based Prompting for Fallacy Recognitioncited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 14; 2024: 51; 2025: 44; 2026: 37.

146 papers.

Sort:

By area:

  1. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 150
  2. 2023CLadder: Assessing Causal Reasoning in Language Modelscited by 107
  3. 2023DyVal: Dynamic Evaluation of Large Language Models for Reasoning Taskscited by 101
  4. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Surveycited by 83
  5. 2024Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Modelscited by 68
  6. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Modelscited by 63
  7. 2024Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Datacited by 63
  8. 2023Integrating Large Language Model for Improved Causal Discoverycited by 54
  9. 2025Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?cited by 38
  10. 2023The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysiscited by 34
  11. 2024How Likely Do LLMs with CoT Mimic Human Reasoning?cited by 34
  12. 2023Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AIcited by 32
38 more
  1. 2024Integrating Large Language Models in Causal Discovery: A Statistical Causal Approachcited by 32
  2. 2024The Essential Role of Causality in Foundation World Models for Embodied AIcited by 32
  3. 2024Causal Inference with Large Language Model: A Surveycited by 32
  4. 2024Causal Prompting: Debiasing Large Language Model Prompting based on Front-Door Adjustmentcited by 30
  5. 2024ALCM: Autonomous LLM-Augmented Causal Discovery Frameworkcited by 29
  6. 2024Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machinacited by 29
  7. 2023CRAB: Assessing the Strength of Causal Relationships Between Real-world Eventscited by 28
  8. 2024Large Language Models and Causal Inference in Collaboration: A Surveycited by 26
  9. 2024Large Language Models for Causal Discovery: Current Landscape and Future Directionscited by 25
  10. 2024Causality for Large Language Modelscited by 25
  11. 2024Large Language Models are Effective Priors for Causal Graph Discoverycited by 24
  12. 2024Cause and Effect: Can Large Language Models Truly Understand Causality?cited by 23
  13. 2024LLMs Are Prone to Fallacies in Causal Inferencecited by 23
  14. 2024Prompting Fairness: Integrating Causality to Debias Large Language Modelscited by 21
  15. 2024Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?cited by 19
  16. 2025Agentic AI Needs a Systems Theorycited by 19
  17. 2024Causal Evaluation of Language Modelscited by 18
  18. 2024End-To-End Causal Effect Estimation from Unstructured Natural Language Datacited by 18
  19. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  20. 2023Towards Causal Foundation Model: on Duality between Causal Inference and Attentioncited by 17
  21. 2025A Survey on Large Language Models for Automated Planningcited by 17
  22. 2025Can LLMs Reason About Program Semantics? A Comprehensive Evaluation of LLMs on Formal Specification Inferencecited by 16
  23. 2023Zero-shot Causal Graph Extrapolation from Text via LLMscited by 15
  24. 2024CLEAR: Can Language Models Really Understand Causal Graphs?cited by 14
  25. 2024A Critical Review of Causal Reasoning Benchmarks for Large Language Modelscited by 14
  26. 2025Abduct, Act, Predict: Scaffolding Causal Inference for Automated Failure Attribution in Multi-Agent Systemscited by 14
  27. 2024Limits of Transformer Language Models on Learning to Compose Algorithmscited by 13
  28. 2024Risks and Opportunities of Open-Source Generative AIcited by 13
  29. 2024CausalEval: Towards Better Causal Reasoning in Language Modelscited by 13
  30. 2024CausalGraph2LLM: Evaluating LLMs for Causal Queriescited by 12
  31. 2024Failure Modes of LLMs for Causal Reasoning on Narrativescited by 12
  32. 2025CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generationcited by 12
  33. 2025Can Large Language Models Help Experimental Design for Causal Discovery?cited by 11
  34. 2024Discovering and Reasoning of Causality in the Hidden World with Large Language Modelscited by 10
  35. 2024CELLO: Causal Evaluation of Large Vision-Language Modelscited by 10
  36. 2024Causal Agent based on Large Language Modelcited by 10
  37. 2025Position: Foundation Models Need Digital Twin Representationscited by 10
  38. 2025CausalAbstain: Enhancing Multilingual LLMs with Causal Reasoning for Trustworthy Abstentioncited by 10

50 most cited of 146 All 146 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Ice Cream Doesn't Cause Drowning: Benchmarking LLMs Against Statistical Pitfalls in Causal Inference
  2. 2025Can Large Language Models Infer Causal Relationships from Real-World Text?
  3. 2024CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMs
  4. 2023Enhancing Causal Reasoning in Large Language Models: A Causal Attribution Model for Precision Fine-Tuning
  5. 2024Counterfactual Causal Inference in Natural Language with Large Language Models
  6. 2026Can Post-Training Transform LLMs into Causal Reasoners?
  7. 2025Causality for Natural Language Processing
  8. 2023LLM4Causal: Democratized Causal Tools for Everyone via Large Language Model