mlsanityData to 7 Oct 2026

NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark

arXiv 2310.18018 ↗First posted 27 Oct 2023cs.CLcited by 339

Not on the map. Why?

Cited by 35 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this position paper, we argue that the classical evaluation on Natural Language Processing (NLP) tasks using annotated benchmarks is in trouble. The worst kind of data contamination happens when a Large Language Model (LLM) is trained on the test split of a benchmark, and then evaluated in the same benchmark. The extent of the problem is unknown, as it is not straightforward to measure. Contamination causes an overestimation of the performance of a contaminated model in a target benchmark and associated task with respect to their non-contaminated counterparts. The consequences can be very harmful, with wrong scientific conclusions being published while other correct ones are discarded. This position paper defines different levels of data contamination and argues for a community effort, including the development of automatic and semi-automatic measures to detect when data from a benchmark was exposed to a model, and suggestions for flagging papers with conclusions that are compromised by data contamination.

Key facts. Cited by 339 papers in the map (data through 7 Oct 2026). Builds on 39 of its 51 references that are in the map. First posted on arXiv 27 Oct 2023.

Builds on

39 of 51 references.

References · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  11. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  12. 2015Teaching Machines to Read and Comprehendcited by 1,039
27 more
  1. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  2. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  3. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  4. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  5. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  6. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  7. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  8. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  9. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  10. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
  11. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 288
  12. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  13. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  14. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  15. 2022Data Contamination: From Memorization to Exploitationcited by 234
  16. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
  17. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 139
  18. 2023Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulnesscited by 130
  19. 2020UnQovering Stereotyping Biases via Underspecified Questionscited by 124
  20. 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 102
  21. 2021Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluationcited by 69
  22. 2023CodeIE: Large Code Generation Models are Better Few-Shot Information Extractorscited by 67
  23. 2020Investigating African-American Vernacular English in Transformer-Based Text Generationcited by 52
  24. 2023An Empirical Study on Information Extraction using Large Language Modelscited by 46
  25. 2023The ROOTS Search Tool: Data Transparency for LLMscited by 30
  26. 2023Data Portraits: Recording Foundation Model Training Datacited by 29
  27. 2023SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2)cited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 99; 2025: 95; 2026: 136.

339 papers.

Sort:

By area:

  1. 2024Qwen2 Technical Reportcited by 2,076
  2. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  3. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 211
  4. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  5. 2024A Careful Examination of Large Language Model Performance on Grade School Arithmeticcited by 188
  6. 2023SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalingcited by 172
  7. 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Modelscited by 150
  8. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  9. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 136
  10. 2024Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questionscited by 106
  11. 2024ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilitiescited by 104
  12. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 98
38 more
  1. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
  2. 2024One Thousand and One Pairs: A "novel" challenge for long-context language modelscited by 80
  3. 2024Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluationcited by 73
  4. 2024Visual Hallucinations of Multi-modal Large Language Modelscited by 68
  5. 2023LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Constructioncited by 57
  6. 2024From LLM to NMT: Advancing Low-Resource Machine Translation with Claudecited by 57
  7. 2026MIRAGE: The Illusion of Visual Understandingcited by 57
  8. 2024From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examplescited by 55
  9. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 52
  10. 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 51
  11. 2023Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Modelscited by 46
  12. 2024GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solvingcited by 44
  13. 2024Latxa: An Open Language Model and Evaluation Suite for Basquecited by 42
  14. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 42
  15. 2024BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrieverscited by 38
  16. 2024In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Misscited by 36
  17. 2024KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Modelscited by 36
  18. 2025The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMscited by 35
  19. 2024ConStat: Performance-Based Contamination Detection in Large Language Modelscited by 33
  20. 2024Training on the Test Task Confounds Evaluation and Emergencecited by 33
  21. 2025A Survey on Data Contamination for Large Language Modelscited by 33
  22. 2024GameArena: Evaluating LLM Reasoning through Live Computer Gamescited by 32
  23. 2024Evading Data Contamination Detection for Language Models is (too) Easycited by 30
  24. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
  25. 2024Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediationcited by 30
  26. 2024Evaluation data contamination in LLMs: how do we measure it and (when) does it matter?cited by 30
  27. 2024VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbationcited by 29
  28. 2025BEARCUBS: A benchmark for computer-using web agentscited by 29
  29. 2024Constructing Domain-Specific Evaluation Sets for LLM-as-a-judgecited by 28
  30. 2025RewardAnything: Generalizable Principle-Following Reward Modelscited by 27
  31. 2024Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptionscited by 26
  32. 2024MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generationcited by 25
  33. 2024Probing Language Models for Pre-training Data Detectioncited by 24
  34. 2024PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Modelscited by 23
  35. 2024Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?cited by 23
  36. 2024Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmarkcited by 23
  37. 2024ASR Error Correction using Large Language Modelscited by 23
  38. 2024Large Language Models are Zero-Shot Next Location Predictorscited by 22

50 most cited of 339 All 339 (JSON)