mlsanityData to 7 Oct 2026

Rethinking Benchmark and Contamination for Language Models with Rephrased Samples

arXiv 2311.04850 ↗First posted 8 Nov 2023cs.CL cs.AIcited by 192

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models are increasingly trained on all the data ever produced by humans. Many have raised concerns about the trustworthiness of public benchmarks due to potential contamination in pre-training or fine-tuning datasets. While most data decontamination efforts apply string matching (e.g., n-gram overlap) to remove benchmark data, we show that these methods are insufficient, and simple variations of test data (e.g., paraphrasing, translation) can easily bypass these decontamination measures. Furthermore, we demonstrate that if such variation of test data is not eliminated, a 13B model can easily overfit a test benchmark and achieve drastically high performance, on par with GPT-4. We validate such observations in widely used benchmarks such as MMLU, GSK8k, and HumanEval. To address this growing risk, we propose a stronger LLM-based decontamination method and apply it to widely used pre-training and fine-tuning datasets, revealing significant previously unknown test overlap. For example, in pre-training sets such as RedPajama-Data-1T and StarCoder-Data, we identified that 8-18% of the HumanEval benchmark overlaps. Interestingly, we also find such contamination in synthetic dataset generated by GPT-3.5/4, suggesting a potential risk of unintentional contamination. We urge the community to adopt stronger decontamination approaches when using public benchmarks. Moreover, we call for the community to actively develop fresh one-time exams to evaluate models accurately. Our decontamination tool is publicly available at https://github.com/lm-sys/llm-decontaminator.

Key facts. Cited by 192 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 6 of its 42 references that are in the map. First posted on arXiv 8 Nov 2023.

Builds on

6 of 42 references.

References · 6

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  6. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 74; 2025: 63; 2026: 52.

192 papers.

Sort:

By area:

  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  2. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  3. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  4. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  5. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  6. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  7. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
  8. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 194
  9. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 188
  10. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 165
  11. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 151
  12. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
38 more
  1. 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 119
  2. 2024On Memorization of Large Language Models in Logical Reasoningcited by 114
  3. 2024BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practicescited by 112
  4. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 106
  5. 2024Benchmarking Benchmark Leakage in Large Language Modelscited by 102
  6. 2025NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelcited by 94
  7. 2024Investigating Data Contamination for Pre-training Language Modelscited by 91
  8. 2024MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixturescited by 73
  9. 2024Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Modelscited by 73
  10. 2024Quantifying Contamination in Evaluating Code Generation Capabilities of Language Modelscited by 62
  11. 2023MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluationcited by 57
  12. 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
  13. 2024Dynamic Evaluation of Large Language Models by Meta Probing Agentscited by 54
  14. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Modelscited by 47
  15. 2023Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Modelscited by 46
  16. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 42
  17. 2024Procedural Knowledge in Pretraining Drives Reasoning in Large Language Modelscited by 41
  18. 2024MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmarkcited by 41
  19. 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
  20. 2025Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoningcited by 38
  21. 2024PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Datacited by 37
  22. 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 34
  23. 2025Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Datasetcited by 34
  24. 2024ConStat: Performance-Based Contamination Detection in Large Language Modelscited by 33
  25. 2024Training on the Test Task Confounds Evaluation and Emergencecited by 33
  26. 2025A Survey on Data Contamination for Large Language Modelscited by 33
  27. 2024Evading Data Contamination Detection for Language Models is (too) Easycited by 30
  28. 2024Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediationcited by 30
  29. 2024Evaluation data contamination in LLMs: how do we measure it and (when) does it matter?cited by 30
  30. 2024Text2Cypher: Bridging Natural Language and Graph Databasescited by 30
  31. 2025Search Arena: Analyzing Search-Augmented LLMscited by 30
  32. 2024VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbationcited by 29
  33. 2025Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluationcited by 29
  34. 2025Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoningcited by 29
  35. 2024CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledgecited by 28
  36. 2024Constructing Domain-Specific Evaluation Sets for LLM-as-a-judgecited by 28
  37. 2024CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeamingcited by 27
  38. 2024Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Modelscited by 26

50 most cited of 192 All 192 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data
  2. 2026Are Large Language Models Truly Smarter Than Humans?
  3. 2024Both Text and Images Leaked! A Systematic Analysis of Data Contamination in Multimodal LLM
  4. 2025Contamination Detection for VLMs using Multi-Modal Semantic Perturbation
  5. 2024Clean Evaluations on Contaminated Visual Language Models
  6. 2025Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment
  7. 2026Calibrating Post-Training Feature Shifts for LLM Data Contamination Detection
  8. 2025Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination