# Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models

arXiv: https://arxiv.org/abs/2402.15938 · HTML: https://mlsanity.com/paper/2402.15938/

## Key facts

- Cited by 122 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 24 Feb 2024.

## Builds on (most cited first, 21 of 21)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [Data Contamination: From Memorization to Exploitation](https://mlsanity.com/paper/2203.08242/) (2022)
- [Proving Test Set Contamination in Black Box Language Models](https://mlsanity.com/paper/2310.17623/) (2023)
- [Rethinking Benchmark and Contamination for Language Models with Rephrased Samples](https://mlsanity.com/paper/2311.04850/) (2023)
- [Don't Make Your LLM an Evaluation Benchmark Cheater](https://mlsanity.com/paper/2311.01964/) (2023)
- [Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks](https://mlsanity.com/paper/2305.10160/) (2023)
- [CodeScore: Evaluating Code Generation by Learning Code Execution](https://mlsanity.com/paper/2301.09043/) (2023)
- [LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Construction](https://mlsanity.com/paper/2312.12343/) (2023)
- [Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model Evaluation](https://mlsanity.com/paper/2309.10677/) (2023)
- [Data Contamination Through the Lens of Time](https://mlsanity.com/paper/2310.10628/) (2023)
- [Competition-Level Problems are Effective LLM Evaluators](https://mlsanity.com/paper/2312.02143/) (2023)
- [Incorporating Domain Knowledge through Task Augmentation for Front-End JavaScript Code Generation](https://mlsanity.com/paper/2208.10091/) (2022)
- [DevEval: Evaluating Code Generation in Practical Software Projects](https://mlsanity.com/paper/2401.06401/) (2024)

## Built on by (most cited first, 30 of 122)

- [LiveBench: A Challenging, Contamination-Limited LLM Benchmark](https://mlsanity.com/paper/2406.19314/) (2024)
- [Preference Leakage: A Contamination Problem in LLM-as-a-judge](https://mlsanity.com/paper/2502.01534/) (2025)
- [Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models](https://mlsanity.com/paper/2404.02936/) (2024)
- [Formal Mathematical Reasoning: A New Frontier in AI](https://mlsanity.com/paper/2412.16075/) (2024)
- [SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)](https://mlsanity.com/paper/2406.17975/) (2024)
- [lmgame-Bench: How Good are LLMs at Playing Games?](https://mlsanity.com/paper/2505.15146/) (2025)
- [Neither Valid nor Reliable? Investigating the Use of LLMs as Judges](https://mlsanity.com/paper/2508.18076/) (2025)
- [M-Longdoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Framework](https://mlsanity.com/paper/2411.06176/) (2024)
- [EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations](https://mlsanity.com/paper/2410.22821/) (2024)
- [VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding](https://mlsanity.com/paper/2505.01481/) (2025)
- [Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoning](https://mlsanity.com/paper/2502.04381/) (2025)
- [EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code](https://mlsanity.com/paper/2505.13004/) (2025)
- [AlgoTune: Can Language Models Speed Up General-Purpose Numerical Programs?](https://mlsanity.com/paper/2507.15887/) (2025)
- [DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph](https://mlsanity.com/paper/2406.17271/) (2024)
- [CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment](https://mlsanity.com/paper/2510.18471/) (2025)
- [DataGen: Unified Synthetic Dataset Generation via Large Language Models](https://mlsanity.com/paper/2406.18966/) (2024)
- [Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination](https://mlsanity.com/paper/2503.04149/) (2025)
- [Automating Expert-Level Medical Reasoning Evaluation of Large Language Models](https://mlsanity.com/paper/2507.07988/) (2025)
- [Are Large Vision Language Models Good Game Players?](https://mlsanity.com/paper/2503.02358/) (2025)
- [Search-Time Data Contamination](https://mlsanity.com/paper/2508.13180/) (2025)
- [Quantifying Generalization Complexity for Large Language Models](https://mlsanity.com/paper/2410.01769/) (2024)
- [Language model developers should report train-test overlap](https://mlsanity.com/paper/2410.08385/) (2024)
- [The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination](https://mlsanity.com/paper/2503.16402/) (2025)
- [MedArabiQ: Benchmarking Large Language Models on Arabic Medical Tasks](https://mlsanity.com/paper/2505.03427/) (2025)
- [LiCoEval: Evaluating LLMs on License Compliance in Code Generation](https://mlsanity.com/paper/2408.02487/) (2024)
- [FAN: Fourier Analysis Networks](https://mlsanity.com/paper/2410.02675/) (2024)
- [Detecting Training Data of Large Language Models via Expectation Maximization](https://mlsanity.com/paper/2410.07582/) (2024)
- [UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models](https://mlsanity.com/paper/2501.13766/) (2025)
- [MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systems](https://mlsanity.com/paper/2505.18943/) (2025)
- [SynAdapt: Learning Adaptive Reasoning in Large Language Models via Synthetic Continuous Chain-of-Thought](https://mlsanity.com/paper/2508.00574/) (2025)

## Abstract (arXiv)

Recent statements about the impressive capabilities of large language models (LLMs) are usually supported by evaluating on open-access benchmarks. Considering the vast size and wide-ranging sources of LLMs' training data, it could explicitly or implicitly include test data, leading to LLMs being more susceptible to data contamination. However, due to the opacity of training data, the black-box access of models, and the rapid growth of synthetic training data, detecting and mitigating data contamination for LLMs faces significant challenges. In this paper, we propose CDD, which stands for Contamination Detection via output Distribution for LLMs. CDD necessitates only the sampled texts to detect data contamination, by identifying the peakedness of LLM's output distribution. To mitigate the impact of data contamination in evaluation, we also present TED: Trustworthy Evaluation via output Distribution, based on the correction of LLM's output distribution. To facilitate this study, we introduce two benchmarks, i.e., DetCon and ComiEval, for data contamination detection and contamination mitigation evaluation tasks. Extensive experimental results show that CDD achieves the average relative improvements of 21.8%-30.2% over other contamination detection approaches in terms of Accuracy, F1 Score, and AUC metrics, and can effectively detect implicit contamination. TED substantially mitigates performance improvements up to 66.9% attributed to data contamination across various contamination setups. In real-world applications, we reveal that ChatGPT exhibits a high potential to suffer from data contamination on HumanEval benchmark.