mlsanityData to 14 Oct 2026

Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions

arXiv 2207.14251 ↗First posted 28 Jul 2022cs.CLcited by 57

Not on the map. Why?

Abstract (arXiv)

Large amounts of training data are one of the major reasons for the high performance of state-of-the-art NLP models. But what exactly in the training data causes a model to make a certain prediction? We seek to answer this question by providing a language for describing how training data influences predictions, through a causal framework. Importantly, our framework bypasses the need to retrain expensive models and allows us to estimate causal effects based on observational data alone. Addressing the problem of extracting factual knowledge from pretrained language models (PLMs), we focus on simple data statistics such as co-occurrence counts and show that these statistics do influence the predictions of PLMs, suggesting that such models rely on shallow heuristics. Our causal framework and our results demonstrate the importance of studying datasets and the benefits of causality for understanding NLP models.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Builds on 47 of its 64 references that are in the map. First posted on arXiv 28 Jul 2022.

Builds on

47 of 64 references.

References · 47

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  6. 2020Extracting Training Data from Large Language Modelscited by 2,189
  7. 2019Language Models as Knowledge Bases?cited by 1,936
  8. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  9. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  10. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  11. 2018Annotation Artifacts in Natural Language Inference Datacited by 671
  12. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
35 more
  1. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 320
  2. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 295
  3. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 243
  4. 2018Stress Test Evaluation for Natural Language Inferencecited by 213
  5. 2020How Context Affects Language Models' Factual Predictionscited by 181
  6. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 178
  7. 2021Counterfactual Memorization in Neural Language Modelscited by 156
  8. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  9. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  10. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
  11. 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
  12. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  13. 2020X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Modelscited by 117
  14. 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 111
  15. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  16. 2019Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Modelcited by 102
  17. 2018Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Informationcited by 97
  18. 2019E-BERT: Efficient-Yet-Effective Entity Embeddings for BERTcited by 96
  19. 2019The emergence of number and syntax units in LSTM language modelscited by 94
  20. 2021The MultiBERTs: BERT Reproductions for Robustness Analysiscited by 92
  21. 2018How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarkscited by 88
  22. 2021Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Testscited by 77
  23. 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 62
  24. 2018Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailmentcited by 59
  25. 2021Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Predictioncited by 59
  26. 2021Frequency Effects on Syntactic Rule Learning in Transformerscited by 59
  27. 2021BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Beliefcited by 57
  28. 2021Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schemacited by 32
  29. 2021Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLPcited by 21
  30. 2022Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Languagecited by 21
  31. 2020Syntactic Search by Examplecited by 20
  32. 2021Causal Distillation for Language Modelscited by 19
  33. 2020An Analysis of Dataset Overlap on Winograd-Style Taskscited by 16
  34. 2021Mediators in Determining what Processing BERT Performs Firstcited by 14
  35. 2021Static Embeddings as Efficient Knowledge Bases?cited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 6; 2023: 19; 2024: 17; 2025: 10; 2026: 5.

57 papers.

Sort:

By area:

  1. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  2. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  3. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  4. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  5. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 178
  6. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 147
  7. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  8. 2023Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4cited by 125
  9. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 103
  10. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  11. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  12. 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 90
38 more
  1. 2024Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Datacited by 85
  2. 2024No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performancecited by 75
  3. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  4. 2023The Bias Amplification Paradox in Text-to-Image Generationcited by 70
  5. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  6. 2023Knowledge is a Region in Weight Space for Fine-tuned Language Modelscited by 58
  7. 2023Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modelingcited by 56
  8. 2024Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Modelscited by 28
  9. 2024Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?cited by 28
  10. 2024Understanding Transformers via N-gram Statisticscited by 26
  11. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 25
  12. 2024Exploring the Impact of Table-to-Text Methods on Augmenting LLM-based Question Answering with Domain Hybrid Datacited by 24
  13. 2024Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Modelscited by 24
  14. 2023The Life Cycle of Knowledge in Big Language Models: A Surveycited by 23
  15. 2025On Linear Representations and Pretraining Data Frequency in Language Modelscited by 23
  16. 2023Impact of Co-occurrence on Factual Knowledge of Large Language Modelscited by 19
  17. 2023DLAMA: A Framework for Curating Culturally Diverse Facts for Probing the Knowledge of Pretrained Language Modelscited by 17
  18. 2025Implicit Reasoning in Transformers is Reasoning through Shortcutscited by 14
  19. 2022Despite "super-human" performance, current LLMs are unsuited for decisions about ethics and safetycited by 13
  20. 2025Robust Data Watermarking in Language Models by Injecting Fictitious Knowledgecited by 12
  21. 2023The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Modelscited by 11
  22. 2024Are Models Biased on Text without Gender-related Language?cited by 10
  23. 2025Tracing Multilingual Factual Knowledge Acquisition in Pretrainingcited by 10
  24. 2022Benchmarking Long-tail Generalization with Likelihood Splitscited by 9
  25. 2023Polyglot or Not? Measuring Multilingual Encyclopedic Knowledge in Foundation Modelscited by 4
  26. 2024Derivational Morphology Reveals Analogical Generalization in Large Language Modelscited by 4
  27. 2023Estimating the Causal Effects of Natural Logic Features in Neural NLI Modelscited by 3
  28. 2024Probing Critical Learning Dynamics of PLMs for Hate Speech Detectioncited by 3
  29. 2025Facts in Stats: Impacts of Pretraining Diversity on Language Model Generalizationcited by 3
  30. 2023What has ChatGPT read? The origins of archaeological citations used by a generative artificial intelligence applicationcited by 2
  31. 2023Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Modelscited by 2
  32. 2025HeQ: a Large and Diverse Hebrew Reading Comprehension Benchmarkcited by 2
  33. 2024Latent Causal Probing: A Formal Perspective on Probing with Causal Models of Datacited by 1
  34. 2024Causal Interventions on Causal Paths: Mapping GPT-2's Reasoning From Syntax to Semanticscited by 1
  35. 2025From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modelingcited by 1
  36. 2025Protoknowledge Shapes Behaviour of LLMs in Downstream Tasks: Memorization and Generalization with Knowledge Graphscited by 1
  37. 2026Truth as a Compression Artifact in Language Model Trainingcited by 1
  38. 2022Underspecification in Language Modeling Tasks: A Causality-Informed Study of Gendered Pronoun Resolutioncited by 0

50 most cited of 57 All 57 (JSON)