mlsanityData to 7 Oct 2026

Large Language Models and Causal Inference in Collaboration: A Survey

arXiv 2403.09606 ↗First posted 14 Mar 2024cs.CL cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Causal inference has shown potential in enhancing the predictive accuracy, fairness, robustness, and explainability of Natural Language Processing (NLP) models by capturing causal relationships among variables. The emergence of generative Large Language Models (LLMs) has significantly impacted various NLP domains, particularly through their advanced reasoning capabilities. This survey focuses on evaluating and improving LLMs from a causal view in the following areas: understanding and improving the LLMs' reasoning capacity, addressing fairness and safety issues in LLMs, complementing LLMs with explanations, and handling multimodality. Meanwhile, LLMs' strong reasoning capacities can in turn contribute to the field of causal inference by aiding causal relationship discovery and causal effect estimations. This review explores the interplay between causal inference frameworks and LLMs from both perspectives, emphasizing their collective potential to further the development of more advanced and equitable artificial intelligence systems.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 67 of its 122 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

67 of 122 references.

References · 67

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2023Qwen Technical Reportcited by 3,273
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  12. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
38 more
  1. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  2. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  3. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  4. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  5. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  6. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 348
  7. 2023Explainability for Large Language Models: A Surveycited by 331
  8. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  9. 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
  10. 2019Evaluating Gender Bias in Machine Translationcited by 247
  11. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
  12. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 177
  13. 2019Certified Robustness to Adversarial Word Substitutionscited by 164
  14. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 146
  15. 2023Can Large Language Models Infer Causation from Correlation?cited by 142
  16. 2020MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radiuscited by 114
  17. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 102
  18. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
  19. 2024Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequencescited by 93
  20. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  21. 2023Can large language models build causal graphs?cited by 72
  22. 2023Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluationcited by 69
  23. 2023Integrating Large Language Model for Improved Causal Discoverycited by 54
  24. 2023BiasAsker: Measuring the Bias in Conversational AI Systemcited by 52
  25. 2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Modelscited by 50
  26. 2023Explore Spurious Correlations at the Concept Level in Language Models for Text Classificationcited by 48
  27. 2023Causal Order: The Key to Leveraging Imperfect Experts in Causal Inferencecited by 46
  28. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  29. 2023MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Taskscited by 44
  30. 2023Causal-Discovery Performance of ChatGPT in the context of Neuropathic Pain Diagnosiscited by 36
  31. 2022Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal Viewcited by 31
  32. 2021Defending Pre-trained Language Models from Adversarial Word Substitutions Without Performance Sacrificecited by 30
  33. 2022Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQAcited by 30
  34. 2022Neuro-Symbolic Procedural Planning with Commonsense Promptingcited by 28
  35. 2023CRAB: Assessing the Strength of Causal Relationships Between Real-world Eventscited by 27
  36. 2023IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositionscited by 25
  37. 2023People Make Better Edits: Measuring the Efficacy of LLM-Generated Counterfactually Augmented Data for Harmful Language Detectioncited by 25
  38. 2023Link-Context Learning for Multimodal LLMscited by 24

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 9; 2026: 2.

26 papers.

Sort:

By area:

  1. 2024Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causalitycited by 42
  2. 2024Large Language Models for UAVs: Current State and Pathways to the Futurecited by 33
  3. 2024AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language Modelscited by 30
  4. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
  5. 2024Causality for Large Language Modelscited by 24
  6. 2025CounterBench: Evaluating and Improving Counterfactual Reasoning in Large Language Modelscited by 19
  7. 2024CausalEval: Towards Better Causal Reasoning in Language Modelscited by 13
  8. 2025MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEscited by 13
  9. 2024Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarioscited by 12
  10. 2025Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberationcited by 11
  11. 2025CausalAbstain: Enhancing Multilingual LLMs with Causal Reasoning for Trustworthy Abstentioncited by 10
  12. 2024The Promises and Pitfalls of Using Language Models to Measure Instruction Quality in Educationcited by 9
14 more
  1. 2024What Do Speech Foundation Models Not Learn About Speech?cited by 8
  2. 2025A Survey on Enhancing Causal Reasoning Ability of Large Language Modelscited by 8
  3. 2024Enhancing Answer Reliability Through Inter-Model Consensus of Large Language Modelscited by 6
  4. 2024Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillationcited by 5
  5. 2024Multimodal Causal Reasoning Benchmark: Challenging Vision Large Language Models to Discern Causal Links Across Modalitiescited by 5
  6. 2025Neuroplasticity and Corruption in Model Mechanisms: A Case Study Of Indirect Object Identificationcited by 3
  7. 2025Neutralizing Bias in LLM Reasoning using Entailment Graphscited by 3
  8. 2024Do LLMs Have the Generalization Ability in Conducting Causal Inference?cited by 2
  9. 2024Unified Causality Analysis Based on the Degrees of Freedomcited by 1
  10. 2025A Technical Exploration of Causal Inference with Hybrid LLM Synthetic Datacited by 1
  11. 2025CRAwDAD: Causal Reasoning Augmentation with Dual-Agent Debatecited by 1
  12. 2026Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunitiescited by 1
  13. 2024DeepFM-Crispr: Prediction of CRISPR On-Target Effects via Deep Learningcited by 0
  14. 2026CaRGo-T: Causal Reasoning Graph-of-Thought improves Multimodal Humor Comprehensioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Causal Inference with Large Language Model: A Survey
  2. 2025Causality for Natural Language Processing
  3. 2023CLadder: Assessing Causal Reasoning in Language Models
  4. 2024A Critical Review of Causal Reasoning Benchmarks for Large Language Models
  5. 2024CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMs
  6. 2025Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
  7. 2025Ice Cream Doesn't Cause Drowning: Benchmarking LLMs Against Statistical Pitfalls in Causal Inference
  8. 2023Enhancing Causal Reasoning in Large Language Models: A Causal Attribution Model for Precision Fine-Tuning