mlsanityData to 7 Oct 2026

Can Foundation Models Talk Causality?

arXiv 2206.10591 ↗First posted 14 Jun 2022cs.AI cs.CL cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

Foundation models are subject to an ongoing heated debate, leaving open the question of progress towards AGI and dividing the community into two camps: the ones who see the arguably impressive results as evidence to the scaling hypothesis, and the others who are worried about the lack of interpretability and reasoning capabilities. By investigating to which extent causal representations might be captured by these large scale language models, we make a humble efforts towards resolving the ongoing philosophical conflicts.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 12 of its 44 references that are in the map. First posted on arXiv 14 Jun 2022.

Builds on

12 of 44 references.

References · 12

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2019Causality for Machine Learningcited by 242
  8. 2020Causal BERT : Language models for causality detection between events expressed in textcited by 22
  9. 2021Interventional Sum-Product Networks: Causal Inference with Tractable Probabilistic Modelscited by 22
  10. 2020Artificial Intelligence is stupid and causal reasoning won't fix itcited by 20
  11. 2018Detecting non-causal artifacts in multivariate linear regression modelscited by 16
  12. 2018The 30-Year Cycle In The AI Debatecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 8; 2025: 8; 2026: 4.

30 papers.

Sort:

By area:

  1. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 348
  2. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Surveycited by 82
  3. 2023Can large language models build causal graphs?cited by 72
  4. 2023Causal Discovery with Language Models as Imperfect Expertscited by 62
  5. 2023Integrating Large Language Model for Improved Causal Discoverycited by 54
  6. 2023Large Language Models Are Not Strong Abstract Reasonerscited by 52
  7. 2025Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?cited by 36
  8. 2024The Essential Role of Causality in Foundation World Models for Embodied AIcited by 32
  9. 2024ALCM: Autonomous LLM-Augmented Causal Discovery Frameworkcited by 29
  10. 2024Large Language Models are Effective Priors for Causal Graph Discoverycited by 23
  11. 2023Causal Structure Learning Supervised by Large Language Modelcited by 21
  12. 2023Masked Images Are Counterfactual Samples for Robust Fine-tuningcited by 20
18 more
  1. 2024Causal Evaluation of Language Modelscited by 18
  2. 2025Position: Foundation Models Need Digital Twin Representationscited by 10
  3. 2024Towards a Benchmark for Causal Business Process Reasoning with LLMscited by 7
  4. 2024Prompting Strategies for Enabling Large Language Models to Infer Causation from Correlationcited by 7
  5. 2023A Survey of Methods, Challenges and Perspectives in Causalitycited by 6
  6. 2023Optimizing Data-driven Causal Discovery Using Knowledge-guided Searchcited by 5
  7. 2025Ice Cream Doesn't Cause Drowning: Benchmarking LLMs Against Statistical Pitfalls in Causal Inferencecited by 5
  8. 2023UPAR: A Kantian-Inspired Prompting Framework for Enhancing Large Language Model Capabilitiescited by 3
  9. 2025Revealing Multimodal Causality with Large Language Modelscited by 3
  10. 2026The Patient is not a Moving Document: A World Model Training Paradigm for Longitudinal EHRcited by 3
  11. 2024Regularized Multi-LLMs Collaboration for Enhanced Score-based Causal Discoverycited by 1
  12. 2025Leveraging Large Language Models for Automated Causal Loop Diagram Generation: Enhancing System Dynamics Modeling through Curated Prompting Techniquescited by 1
  13. 2025CARE: Turning LLMs Into Causal Reasoning Expertcited by 1
  14. 2025Sequential Causal Discovery with Noisy Language Model Priorscited by 0
  15. 2025Boosting Process-Correct CoT Reasoning by Modeling Solvability of Multiple-Choice QAcited by 0
  16. 2026DMCD: Semantic-Statistical Framework for Causal Discoverycited by 0
  17. 2026CausalSteward: An Agentic Divide-Conquer-Combine Copilot for Causal Discoverycited by 0
  18. 2026Externalized CPDAG Summaries Improve LLM Causal Deductioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024CLEAR: Can Language Models Really Understand Causal Graphs?
  2. 2025Causality can systematically address the monsters under the bench(marks)
  3. 2025Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learning
  4. 2025Do Large Language Models Reason Causally Like Us? Even Better?
  5. 2025What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models
  6. 2024Cause and Effect: Can Large Language Models Truly Understand Causality?
  7. 2023Can Large Language Models Infer Causation from Correlation?
  8. 2023Can We Utilize Pre-trained Language Models within Causal Discovery Algorithms?