mlsanityData to 7 Oct 2026

Investigating Data Contamination in Modern Benchmarks for Large Language Models

arXiv 2311.09783 ↗First posted 16 Nov 2023cs.CL cs.AIcited by 136

Not on the map. Why?

Cited by 23 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent observations have underscored a disparity between the inflated benchmark scores and the actual performance of LLMs, raising concerns about potential contamination of evaluation benchmarks. This issue is especially critical for closed-source models and certain open-source models where training data transparency is lacking. In this paper we study data contamination by proposing two methods tailored for both open-source and proprietary LLMs. We first introduce a retrieval-based system to explore potential overlaps between evaluation benchmarks and pretraining corpora. We further present a novel investigation protocol named Testset Slot Guessing (textitTS-Guessing), applicable to both open and proprietary models. This approach entails masking a wrong answer in a multiple-choice question and prompting the model to fill in the gap. Additionally, it involves obscuring an unlikely word in an evaluation example and asking the model to produce it. We find that certain commercial LLMs could surprisingly guess the missing option in various test sets. Specifically, in the TruthfulQA benchmark, we find that LLMs exhibit notable performance improvement when provided with additional metadata in the benchmark. Further, in the MMLU benchmark, ChatGPT and GPT-4 demonstrated an exact match rate of 52% and 57%, respectively, in guessing the missing options in benchmark test data. We hope these results underscore the need for more robust evaluation methodologies and benchmarks in the field.

Key facts. Cited by 136 papers in the map (data through 7 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 16 Nov 2023.

Builds on

33 of 46 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  11. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
21 more
  1. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  2. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  3. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  4. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  5. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  6. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  7. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  8. 2023Proving Test Set Contamination in Black Box Language Modelscited by 227
  9. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  10. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 139
  11. 2023Skywork: A More Open Bilingual Foundation Modelcited by 110
  12. 2022Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Modelscited by 90
  13. 2023Can we trust the evaluation on ChatGPT?cited by 63
  14. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  15. 2021Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representationscited by 60
  16. 2023Data Contamination Through the Lens of Timecited by 39
  17. 2023Pretraining on the Test Set Is All You Needcited by 33
  18. 2023The ROOTS Search Tool: Data Transparency for LLMscited by 30
  19. 2022FALCON: Fast Visual Concept Learning by Integrating Images, Linguistic descriptions, and Conceptual Relationscited by 20
  20. 2022MPT: Mesh Pre-Training with Transformers for Human Pose and Mesh Reconstructioncited by 11
  21. 2023GAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data Explorationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 28; 2025: 53; 2026: 55.

136 papers.

Sort:

By area:

  1. 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 254
  2. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 242
  3. 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 227
  4. 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 170
  5. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  6. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 119
  7. 2024Benchmarking Benchmark Leakage in Large Language Modelscited by 102
  8. 2025HalluLens: LLM Hallucination Benchmarkcited by 88
  9. 2026MIRAGE: The Illusion of Visual Understandingcited by 57
  10. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 54
  11. 2025Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gapcited by 41
  12. 2025Neither Valid nor Reliable? Investigating the Use of LLMs as Judgescited by 37
38 more
  1. 2025LLM-as-a-Prophet: Understanding Predictive Intelligence with Prophet Arenacited by 36
  2. 2024ConStat: Performance-Based Contamination Detection in Large Language Modelscited by 33
  3. 2024Evading Data Contamination Detection for Language Models is (too) Easycited by 30
  4. 2026LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfacescited by 29
  5. 2026When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturationcited by 29
  6. 2024DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graphcited by 26
  7. 2024Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptionscited by 26
  8. 2024TRUCE: Private Benchmarking to Prevent Contamination and Improve Comparative Evaluation of LLMscited by 24
  9. 2025Llama-3.1-FoundationAI-SecurityLLM-8B-Instruct Technical Reportcited by 21
  10. 2024PhyloLM : Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarkscited by 18
  11. 2024Holistic Safety and Responsibility Evaluations of Advanced AI Modelscited by 18
  12. 2024Eureka: Evaluating and Understanding Large Foundation Modelscited by 18
  13. 2025LLM-Safety Evaluations Lack Robustnesscited by 16
  14. 2025Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmarkcited by 16
  15. 2024On Catastrophic Inheritance of Large Foundation Modelscited by 15
  16. 2024TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Modelscited by 15
  17. 2025MedArabiQ: Benchmarking Large Language Models on Arabic Medical Taskscited by 15
  18. 2024How Much Can We Forget about Data Contamination?cited by 14
  19. 2024Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdoutscited by 13
  20. 2025UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Modelscited by 13
  21. 2025The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contaminationcited by 13
  22. 2024A Comprehensive Survey of Contamination Detection Methods in Large Language Modelscited by 12
  23. 2024MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofscited by 12
  24. 2025The Automated LLM Speedrunning Benchmark: Reproducing NanoGPT Improvementscited by 12
  25. 2024PingPong: A Benchmark for Role-Playing Language Models with User Emulation and Multi-Model Evaluationcited by 10
  26. 2025EngiBench: A Benchmark for Evaluating Large Language Models on Engineering Problem Solvingcited by 10
  27. 2024The Mosaic Memory of Large Language Modelscited by 9
  28. 2024Both Text and Images Leaked! A Systematic Analysis of Data Contamination in Multimodal LLMcited by 9
  29. 2025A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directionscited by 9
  30. 2026SWE-Router: Routing in Multi-turn Agentic Software Engineering Taskscited by 9
  31. 2025NetArena: Dynamic Benchmarks for AI Agents in Network Automationcited by 8
  32. 2025A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gapcited by 8
  33. 2025On The Fragility of Benchmark Contamination Detection in Reasoning Modelscited by 8
  34. 2024Mind the Data Gap: Bridging LLMs to Enterprise Data Integrationcited by 7
  35. 2025STEER-ME: Assessing the Microeconomic Reasoning of Large Language Modelscited by 7
  36. 2025Confidence in Large Language Model Evaluation: A Bayesian Approach to Limited-Sample Challengescited by 7
  37. 2025Federated In-Context Learning: Iterative Refinement for Improved Answer Qualitycited by 7
  38. 2025Pretraining on the Test Set Is No Longer All You Need: A Debate-Driven Approach to QA Benchmarkscited by 7

50 most cited of 136 All 136 (JSON)