mlsanityData to 7 Oct 2026

Data Contamination: From Memorization to Exploitation

arXiv 2203.08242 ↗First posted 15 Mar 2022cs.CL cs.LGcited by 234

Not on the map. Why?

Cited by 25 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Pretrained language models are typically trained on massive web-based datasets, which are often "contaminated" with downstream test sets. It is not clear to what extent models exploit the contaminated data for downstream tasks. We present a principled method to study this question. We pretrain BERT models on joint corpora of Wikipedia and labeled downstream datasets, and fine-tune them on the relevant task. Comparing performance between samples seen and unseen during pretraining enables us to define and quantify levels of memorization and exploitation. Experiments with two models and three downstream tasks show that exploitation exists in some cases, but in others the models memorize the contaminated data, but do not exploit it. We show that these two measures are affected by different factors such as the number of duplications of the contaminated data and the model size. Our results highlight the importance of analyzing massive web-scale datasets to verify that progress in NLP is obtained by better language understanding and not better data exploitation.

Key facts. Cited by 234 papers in the map (data through 7 Oct 2026). Builds on 21 of its 34 references that are in the map. First posted on arXiv 15 Mar 2022.

Builds on

21 of 34 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  8. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  9. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  10. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  11. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  12. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
9 more
  1. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  2. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  3. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
  4. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  5. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  6. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  7. 2021PPT: Pre-trained Prompt Tuning for Few-shot Learningcited by 232
  8. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  9. 2021On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependenciescited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 46; 2024: 79; 2025: 48; 2026: 55.

234 papers.

Sort:

By area:

  1. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  2. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  3. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 243
  4. 2023Proving Test Set Contamination in Black Box Language Modelscited by 227
  5. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  6. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  7. 2024A Careful Examination of Large Language Model Performance on Grade School Arithmeticcited by 188
  8. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 139
  9. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  10. 2023Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4cited by 123
  11. 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 119
  12. 2023GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extractioncited by 116
38 more
  1. 2024On Memorization of Large Language Models in Logical Reasoningcited by 114
  2. 2023Large Language Models as Analogical Reasonerscited by 107
  3. 2024Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questionscited by 106
  4. 2024ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilitiescited by 104
  5. 2024Benchmarking Benchmark Leakage in Large Language Modelscited by 102
  6. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 100
  7. 2023DyVal: Dynamic Evaluation of Large Language Models for Reasoning Taskscited by 100
  8. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  9. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 98
  10. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debatecited by 92
  11. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
  12. 2024Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Datacited by 83
  13. 2023Eight Things to Know about Large Language Modelscited by 73
  14. 2024SpecRover: Code Intent Extraction via LLMscited by 71
  15. 2024Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracycited by 66
  16. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  17. 2024Quantifying Contamination in Evaluating Code Generation Capabilities of Language Modelscited by 62
  18. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Modelscited by 61
  19. 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
  20. 2026MIRAGE: The Illusion of Visual Understandingcited by 57
  21. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 54
  22. 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 51
  23. 2024LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theorycited by 44
  24. 2023Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergencecited by 42
  25. 2023A General-Purpose Self-Supervised Model for Computational Pathologycited by 42
  26. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 42
  27. 2022Prompting Language Models for Linguistic Structurecited by 39
  28. 2023Data Contamination Through the Lens of Timecited by 39
  29. 2024Proving membership in LLM pretraining data via data watermarkscited by 38
  30. 2024If in a Crowdsourced Data Annotation Pipeline, a GPT-4cited by 38
  31. 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
  32. 2023Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?cited by 36
  33. 2023User Inference Attacks on Large Language Modelscited by 36
  34. 2023Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMscited by 34
  35. 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 34
  36. 2022Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Biascited by 33
  37. 2024Training on the Test Task Confounds Evaluation and Emergencecited by 33
  38. 2023Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayingscited by 32

50 most cited of 234 All 234 (JSON)