mlsanityData to 14 Oct 2026

Benchmarking Benchmark Leakage in Large Language Models

arXiv 2404.18824 ↗First posted 29 Apr 2024cs.CL cs.AI cs.LGcited by 106

Not on the map. Why?

Cited by 30 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Amid the expanding use of pre-training data, the phenomenon of benchmark dataset leakage has become increasingly prominent, exacerbated by opaque training processes and the often undisclosed inclusion of supervised data in contemporary Large Language Models (LLMs). This issue skews benchmark effectiveness and fosters potentially unfair comparisons, impeding the field's healthy development. To address this, we introduce a detection pipeline utilizing Perplexity and N-gram accuracy, two simple and scalable metrics that gauge a model's prediction precision on benchmark, to identify potential data leakages. By analyzing 31 LLMs under the context of mathematical reasoning, we reveal substantial instances of training even test set misuse, resulting in potentially unfair comparisons. These findings prompt us to offer several recommendations regarding model documentation, benchmark setup, and future evaluations. Notably, we propose the "Benchmark Transparency Card" to encourage clear documentation of benchmark utilization, promoting transparency and healthy developments of LLMs. we have made our leaderboard, pipeline implementation, and model predictions publicly available, fostering future research.

Key facts. Cited by 106 papers in the map (data through 14 Oct 2026). Builds on 51 of its 63 references that are in the map. First posted on arXiv 29 Apr 2024.

Builds on

51 of 63 references.

References · 51

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  8. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  11. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  12. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
38 more
  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  2. 2023Qwen Technical Reportcited by 3,344
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  5. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  6. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  7. 2018Datasheets for Datasetscited by 1,386
  8. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  9. 2018Model Cards for Model Reportingcited by 1,080
  10. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  11. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  12. 2024Yi: Open Foundation Models by 01.AIcited by 719
  13. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  14. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  15. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  16. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
  17. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  18. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  19. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  20. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 252
  21. 2022Data Contamination: From Memorization to Exploitationcited by 236
  22. 2023Proving Test Set Contamination in Black Box Language Modelscited by 230
  23. 2023Rethinking Benchmark and Contamination for Language Models with Rephrased Samplescited by 195
  24. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  25. 2022Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AIcited by 171
  26. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 159
  27. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 142
  28. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 138
  29. 2023Skywork: A More Open Bilingual Foundation Modelcited by 112
  30. 2023DyVal: Dynamic Evaluation of Large Language Models for Reasoning Taskscited by 101
  31. 2024Investigating Data Contamination for Pre-training Language Modelscited by 91
  32. 2023The Foundation Model Transparency Indexcited by 66
  33. 2024Dynamic Evaluation of Large Language Models by Meta Probing Agentscited by 54
  34. 2020Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Trainingcited by 50
  35. 2023Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model Evaluationcited by 48
  36. 2023Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Modelscited by 47
  37. 2024Nemotron-4 15B Technical Reportcited by 36
  38. 2024Reformatted Alignmentcited by 15

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 33; 2025: 45; 2026: 28.

106 papers.

Sort:

By area:

  1. 2025LIMO: Less is More for Reasoningcited by 498
  2. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 284
  3. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 224
  4. 2024A Careful Examination of Large Language Model Performance on Grade School Arithmeticcited by 191
  5. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 158
  6. 2025xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluationscited by 120
  7. 2024On Memorization of Large Language Models in Logical Reasoningcited by 117
  8. 2024ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilitiescited by 110
  9. 2024NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samplescited by 97
  10. 2025Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contaminationcited by 82
  11. 2024OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AIcited by 71
  12. 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 60
38 more
  1. 2025MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoningcited by 59
  2. 2024EXAONE 3.5: Series of Large Language Models for Real-world Use Casescited by 53
  3. 2025MegaMath: Pushing the Limits of Open Math Corporacited by 51
  4. 2024BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Datacited by 50
  5. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computationcited by 49
  6. 2025UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Modelscited by 48
  7. 2024HelloBench: Evaluating Long Text Generation Capabilities of Large Language Modelscited by 44
  8. 2025Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gapcited by 43
  9. 2024Adaptive In-conversation Team Building for Language Model Agentscited by 39
  10. 2024GLBench: A Comprehensive Benchmark for Graph with Large Language Modelscited by 36
  11. 2024ConStat: Performance-Based Contamination Detection in Large Language Modelscited by 33
  12. 2024Not All LLM Reasoners Are Created Equalcited by 30
  13. 2024RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Contentcited by 26
  14. 2024Training on the Benchmark Is Not All You Needcited by 25
  15. 2024DataGen: Unified Synthetic Dataset Generation via Large Language Modelscited by 23
  16. 2025JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language Modelscited by 22
  17. 2025Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarkingcited by 21
  18. 2026Large Language Model Reasoning Failurescited by 21
  19. 2025Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboardscited by 20
  20. 2024Embedding Trajectory for Out-of-Distribution Detection in Mathematical Reasoningcited by 18
  21. 2024CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMscited by 18
  22. 2026TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Modelscited by 18
  23. 2024Autonomous Agents for Collaborative Task under Information Asymmetrycited by 16
  24. 2024Probabilistic Medical Predictions of Large Language Modelscited by 16
  25. 2025MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Sciencecited by 15
  26. 2025Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoningcited by 14
  27. 2025Scaling Physical Reasoning with the PHYSICS Datasetcited by 13
  28. 2025UNO-Bench: A Unified Benchmark for Exploring the Compositional Law Between Uni-modal and Omni-modal in Omni Modelscited by 13
  29. 2026Benchmark^2: Systematic Evaluation of LLM Benchmarkscited by 13
  30. 2024Are Large Language Models Memorizing Bug Benchmarks?cited by 12
  31. 2025BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Textcited by 12
  32. 2024DICE: Detecting In-distribution Contamination in LLM's Fine-tuning Phase for Math Reasoningcited by 11
  33. 2024Geospatial foundation models for image analysis: evaluating and enhancing NASA-IBM Prithvi's domain adaptabilitycited by 11
  34. 2024Data for Mathematical Copilots: Better Ways of Presenting Proofs for Machine Learningcited by 11
  35. 2024VideoCogQA: A Controllable Benchmark for Evaluating Cognitive Abilities in Video-Language Modelscited by 10
  36. 2026LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematicscited by 10
  37. 2026Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarkingcited by 10
  38. 2025Evaluating the Sensitivity of LLMs to Prior Contextcited by 9

50 most cited of 106 All 106 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progress
  2. 2025LastingBench: Defend Benchmarks Against Knowledge Leakage
  3. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics
  4. 2025Mapping Overlaps in Benchmarks through Perplexity in the Wild
  5. 2026Latent Performance Profiling of Large Language Models
  6. 2025BELL: Benchmarking the Explainability of Large Language Models
  7. 2026Benchmark Shadows: Data Alignment, Parameter Footprints, and Generalization in Large Language Models
  8. 2024A Probabilistic Perspective on Unlearning and Alignment for Large Language Models