mlsanityData to 7 Oct 2026

Don't Make Your LLM an Evaluation Benchmark Cheater

arXiv 2311.01964 ↗First posted 3 Nov 2023cs.CL cs.AIcited by 194

Not on the map. Why?

Cited by 27 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models~(LLMs) have greatly advanced the frontiers of artificial intelligence, attaining remarkable improvement in model capacity. To assess the model performance, a typical approach is to construct evaluation benchmarks for measuring the ability level of LLMs in different aspects. Despite that a number of high-quality benchmarks have been released, the concerns about the appropriate use of these benchmarks and the fair comparison of different models are increasingly growing. Considering these concerns, in this paper, we discuss the potential risk and impact of inappropriately using evaluation benchmarks and misleadingly interpreting the evaluation results. Specially, we focus on a special issue that would lead to inappropriate evaluation, ie benchmark leakage, referring that the data related to evaluation sets is occasionally used for model training. This phenomenon now becomes more common since pre-training data is often prepared ahead of model test. We conduct extensive experiments to study the effect of benchmark leverage, and find that it can dramatically boost the evaluation results, which would finally lead to an unreliable assessment of model performance. To improve the use of existing evaluation benchmarks, we finally present several guidelines for both LLM developers and benchmark maintainers. We hope this work can draw attention to appropriate training and evaluation of LLMs.

Key facts. Cited by 194 papers in the map (data through 7 Oct 2026). Builds on 31 of its 44 references that are in the map. First posted on arXiv 3 Nov 2023.

Builds on

31 of 44 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  9. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  12. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
19 more
  1. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  2. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  3. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  4. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  5. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  6. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  7. 2018CoQA: A Conversational Question Answering Challengecited by 674
  8. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  9. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  10. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  11. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 536
  12. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  13. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  14. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  15. 2023Proving Test Set Contamination in Black Box Language Modelscited by 227
  16. 2015The Goldilocks Principle: Reading Children's Books with Explicit Memory Representationscited by 108
  17. 2018A Span-Extraction Dataset for Chinese Machine Reading Comprehensioncited by 84
  18. 2019Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehensioncited by 80
  19. 2023Can we trust the evaluation on ChatGPT?cited by 63

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 94; 2025: 62; 2026: 27.

194 papers.

Sort:

By area:

  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  2. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  3. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluationcited by 293
  4. 2024Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performancecited by 179
  5. 2023SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalingcited by 172
  6. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 153
  7. 2024ProSA: Assessing and Understanding the Prompt Sensitivity of LLMscited by 145
  8. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  9. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 136
  10. 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 119
  11. 2024Benchmarking Benchmark Leakage in Large Language Modelscited by 102
  12. 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 102
38 more
  1. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 98
  2. 2024NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samplescited by 94
  3. 2025Establishing Best Practices for Building Rigorous Agentic Benchmarkscited by 94
  4. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
  5. 2024LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256Kcited by 80
  6. 2024Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluationcited by 73
  7. 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasonerscited by 73
  8. 2024Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?cited by 68
  9. 2024Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligencecited by 65
  10. 2024Are Your LLMs Capable of Stable Reasoning?cited by 65
  11. 2023MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Taskscited by 57
  12. 2024Dynamic Evaluation of Large Language Models by Meta Probing Agentscited by 54
  13. 2024What's Wrong with Your Code Generated by Large Language Models? An Extensive Studycited by 52
  14. 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 51
  15. 2024Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklistcited by 48
  16. 2025The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reasoncited by 47
  17. 2023Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Modelscited by 46
  18. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 42
  19. 2023Fake Alignment: Are LLMs Really Aligned Well?cited by 39
  20. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 38
  21. 2024KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Modelscited by 36
  22. 2024LLMArena: Assessing Capabilities of Large Language Models in Dynamic Multi-Agent Environmentscited by 35
  23. 2024ConStat: Performance-Based Contamination Detection in Large Language Modelscited by 33
  24. 2024Training on the Test Task Confounds Evaluation and Emergencecited by 33
  25. 2024Large Language Models as Biomedical Hypothesis Generators: A Comprehensive Evaluationcited by 33
  26. 2024Examining the robustness of LLM evaluation to the distributional assumptions of benchmarkscited by 31
  27. 2024Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediationcited by 30
  28. 2024Easy Problems That LLMs Get Wrongcited by 29
  29. 2024VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbationcited by 29
  30. 2024Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machinacited by 29
  31. 2023An In-depth Look at Gemini's Language Abilitiescited by 28
  32. 2024LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languagescited by 28
  33. 2024Cofca: A Step-Wise Counterfactual Multi-hop QA benchmarkcited by 26
  34. 2024DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graphcited by 26
  35. 2024Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptionscited by 26
  36. 2025A Survey of Multimodal Hallucination Evaluation and Detectioncited by 26
  37. 2023Competition-Level Problems are Effective LLM Evaluatorscited by 25
  38. 2024Training on the Benchmark Is Not All You Needcited by 24

50 most cited of 194 All 194 (JSON)