mlsanityData to 14 Oct 2026

Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks

arXiv 2305.10160 ↗First posted 17 May 2023cs.CL cs.AIcited by 142

Not on the map. Why?

Abstract (arXiv)

Data contamination has become prevalent and challenging with the rise of models pretrained on large automatically-crawled corpora. For closed models, the training data becomes a trade secret, and even for open models, it is not trivial to detect contamination. Strategies such as leaderboards with hidden answers, or using test data which is guaranteed to be unseen, are expensive and become fragile with time. Assuming that all relevant actors value clean test data and will cooperate to mitigate data contamination, what can be done? We propose three strategies that can make a difference: (1) Test data made public should be encrypted with a public key and licensed to disallow derivative distribution; (2) demand training exclusion controls from closed API holders, and protect your test data by refusing to evaluate without them; (3) avoid data which appears with its solution on the internet, and release the web-page context of internet-derived data along with the data. These strategies are practical and can be effective in preventing data contamination.

Key facts. Cited by 142 papers in the map (data through 14 Oct 2026). Builds on 35 of its 59 references that are in the map. First posted on arXiv 17 May 2023.

Builds on

35 of 59 references.

References · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2020Extracting Training Data from Large Language Modelscited by 2,189
  8. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  9. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  10. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  11. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  12. 2021Robust fine-tuning of zero-shot modelscited by 797
23 more
  1. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  2. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  3. 2023Extracting Training Data from Diffusion Modelscited by 732
  4. 2023A Watermark for Large Language Modelscited by 725
  5. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  6. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  7. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  8. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  9. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  10. 2023Can AI-Generated Text be Reliably Detected?cited by 386
  11. 2023Evaluating Verifiability in Generative Search Enginescited by 303
  12. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  13. 2022Data Contamination: From Memorization to Exploitationcited by 236
  14. 2023Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4cited by 125
  15. 2017Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasmcited by 118
  16. 2022Fusing finetuned models for better pretrainingcited by 110
  17. 2020Compressing Large-Scale Transformer-Based Models: A Case Study on BERTcited by 96
  18. 2020Contrastive Distillation on Intermediate Representations for Language Model Compressioncited by 57
  19. 2014Effective Use of Word Order for Text Categorization with Convolutional Neural Networkscited by 40
  20. 2023The ROOTS Search Tool: Data Transparency for LLMscited by 31
  21. 2023Data Portraits: Recording Foundation Model Training Datacited by 30
  22. 2023ThoughtSource: A central hub for large language model reasoning datacited by 23
  23. 2021GENIE: Toward Reproducible and Standardized Human Evaluation for Text Generationcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 20; 2024: 48; 2025: 24; 2026: 50.

142 papers.

Sort:

By area:

  1. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  2. 2023Challenges and Applications of Large Language Modelscited by 254
  3. 2024A Careful Examination of Large Language Model Performance on Grade School Arithmeticcited by 191
  4. 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Modelscited by 152
  5. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 147
  6. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 138
  7. 2025MMVU: Measuring Expert-Level Multi-Discipline Video Understandingcited by 137
  8. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  9. 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 122
  10. 2024BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practicescited by 114
  11. 2024ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilitiescited by 110
  12. 2024Benchmarking Benchmark Leakage in Large Language Modelscited by 106
38 more
  1. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 102
  2. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 101
  3. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 99
  4. 2023Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?cited by 88
  5. 2024One Thousand and One Pairs: A "novel" challenge for long-context language modelscited by 80
  6. 2024Visual Hallucinations of Multi-modal Large Language Modelscited by 68
  7. 2024A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chainscited by 65
  8. 2023LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Constructioncited by 58
  9. 2023An Open Source Data Contamination Report for Large Language Modelscited by 51
  10. 2024A Performance Study of LLM-Generated Code on Leetcodecited by 49
  11. 2023Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model Evaluationcited by 48
  12. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 44
  13. 2023A General-Purpose Self-Supervised Model for Computational Pathologycited by 42
  14. 2024Prototypical Information Bottlenecking and Disentangling for Multimodal Cancer Survival Predictioncited by 41
  15. 2023Data Contamination Through the Lens of Timecited by 40
  16. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 39
  17. 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
  18. 2024Evaluation data contamination in LLMs: how do we measure it and (when) does it matter?cited by 32
  19. 2024Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediationcited by 30
  20. 2024VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbationcited by 29
  21. 2023Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answeringcited by 28
  22. 2023CRAB: Assessing the Strength of Causal Relationships Between Real-world Eventscited by 28
  23. 2024LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languagescited by 28
  24. 2024Text Embedding Inversion Security for Multilingual Language Modelscited by 26
  25. 2024MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generationcited by 26
  26. 2024Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptionscited by 26
  27. 2024FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culturecited by 25
  28. 2025On Benchmarking Human-Like Intelligence in Machinescited by 25
  29. 2025Re-evaluating Theory of Mind evaluation in large language modelscited by 23
  30. 2024Data Contamination Report from the 2024 CONDA Shared Taskcited by 20
  31. 2024Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Modelscited by 20
  32. 2024FinDVer: Explainable Claim Verification over Long and Hybrid-Content Financial Documentscited by 19
  33. 2024Towards Data Contamination Detection for Modern Large Language Models: Limitations, Inconsistencies, and Oracle Challengescited by 18
  34. 2023Large Language Models are Fixated by Red Herrings: Exploring Creative Problem Solving and Einstellung Effect using the Only Connect Wall Datasetcited by 17
  35. 2024Robust Pronoun Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased?cited by 17
  36. 2024A Notion of Complexity for Theory of Mind via Discrete World Modelscited by 16
  37. 2025How Well Does GPT-4o Understand Vision? Evaluating Multimodal Foundation Models on Standard Computer Vision Taskscited by 14
  38. 2026Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJackcited by 14

50 most cited of 142 All 142 (JSON)