# How Contaminated Is Your Benchmark? Quantifying Dataset Leakage in Large Language Models with Kernel Divergence

arXiv: https://arxiv.org/abs/2502.00678 · HTML: https://mlsanity.com/paper/2502.00678/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 37 references that are in the map.
- First posted on arXiv 2 Feb 2025.

## Builds on (most cited first, 29 of 29)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Detecting Pretraining Data from Large Language Models](https://mlsanity.com/paper/2310.16789/) (2023)
- [Membership Inference Attacks against Language Models via Neighbourhood Comparison](https://mlsanity.com/paper/2305.18462/) (2023)
- [Data Contamination: From Memorization to Exploitation](https://mlsanity.com/paper/2203.08242/) (2022)
- [Proving Test Set Contamination in Black Box Language Models](https://mlsanity.com/paper/2310.17623/) (2023)
- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [Rethinking Benchmark and Contamination for Language Models with Rephrased Samples](https://mlsanity.com/paper/2311.04850/) (2023)
- [Do Membership Inference Attacks Work on Large Language Models?](https://mlsanity.com/paper/2402.07841/) (2024)
- [Don't Make Your LLM an Evaluation Benchmark Cheater](https://mlsanity.com/paper/2311.01964/) (2023)
- [Time Travel in LLMs: Tracing Data Contamination in Large Language Models](https://mlsanity.com/paper/2308.08493/) (2023)
- [Benchmark Data Contamination of Large Language Models: A Survey](https://mlsanity.com/paper/2406.04244/) (2024)
- [LLM Dataset Inference: Did you train on my dataset?](https://mlsanity.com/paper/2406.06443/) (2024)
- [Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models](https://mlsanity.com/paper/2404.02936/) (2024)
- [DE-COP: Detecting Copyrighted Content in Language Models Training Data](https://mlsanity.com/paper/2402.09910/) (2024)
- [Blind Baselines Beat Membership Inference Attacks for Foundation Models](https://mlsanity.com/paper/2406.16201/) (2024)
- [ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods](https://mlsanity.com/paper/2406.15968/) (2024)
- [SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)](https://mlsanity.com/paper/2406.17975/) (2024)
- [Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model Evaluation](https://mlsanity.com/paper/2309.10677/) (2023)
- [ConStat: Performance-Based Contamination Detection in Large Language Models](https://mlsanity.com/paper/2405.16281/) (2024)
- [Data Contamination Calibration for Black-box LLMs](https://mlsanity.com/paper/2405.11930/) (2024)
- [Fine-tuning can Help Detect Pretraining Data from Large Language Models](https://mlsanity.com/paper/2410.10880/) (2024)

## Built on by (most cited first, 15 of 15)

- [BeyondBench: Contamination-Resistant Evaluation of Reasoning in Language Models](https://mlsanity.com/paper/2509.24210/) (2025)
- [Evaluating the Robustness of Large Language Model Safety Guardrails Against Adversarial Attacks](https://mlsanity.com/paper/2511.22047/) (2025)
- [Membership Inference Attacks on Tokenizers of Large Language Models](https://mlsanity.com/paper/2510.05699/) (2025)
- [TSFMAudit: Data Contamination Auditing in Forecasting Time Series Foundation Models](https://mlsanity.com/paper/2605.26161/) (2026)
- [One Patient, Many Contexts: Scaling Medical AI with Contextual Intelligence](https://mlsanity.com/paper/2506.10157/) (2025)
- [ArenaBencher: Automatic Benchmark Evolution via Multi-Model Competitive Evaluation](https://mlsanity.com/paper/2510.08569/) (2025)
- [Contamination Detection for VLMs using Multi-Modal Semantic Perturbation](https://mlsanity.com/paper/2511.03774/) (2025)
- [Unmasking Reasoning Processes: A Process-aware Benchmark for Evaluating Structural Mathematical Reasoning in LLMs](https://mlsanity.com/paper/2602.00564/) (2026)
- [When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors](https://mlsanity.com/paper/2604.21255/) (2026)
- [Is Domain Adaptation Always Helpful? A Frozen-Backbone Study of Cross-Domain Sentiment Transfer](https://mlsanity.com/paper/2607.05937/) (2026)
- [From Unfamiliar to Familiar: Detecting Pre-training Data via Gradient Deviations in Large Language Models](https://mlsanity.com/paper/2603.04828/) (2026)
- [Provable Joint Decontamination for Benchmarking Multiple Large Language Models](https://mlsanity.com/paper/2605.21543/) (2026)
- [LaRA: Layer-wise Representation Analysis for Detecting Data Contamination in RL Post-Training](https://mlsanity.com/paper/2605.29888/) (2026)
- [MC-PDD: Masked Corpus-Level Pretraining Data Detection for Black-Box Large Language Models](https://mlsanity.com/paper/2606.07996/) (2026)
- [WildTrace: Evaluating Training Data Detection in Realistic Large-Scale LLM Post-Training](https://mlsanity.com/paper/2610.16152/) (2026)

## Abstract (arXiv)

Dataset contamination, where evaluation datasets overlap with pre-training corpora, inflates performance metrics and undermines the reliability of model evaluations. Measuring dataset contamination thus becomes essential to ensure that performance evaluations genuinely reflect a model's ability to generalize to unseen data, rather than relying on memorized examples. To address this problem, we propose Kernel Divergence Score (KDS), a novel method that evaluates dataset contamination by computing the divergence between the kernel similarity matrix of sample embeddings, before and after fine-tuning on the benchmark dataset. Leveraging the insight that fine-tuning affects unseen samples more significantly than seen ones, KDS provides a reliable measure of contamination. Through extensive experiments on controlled contamination scenarios, KDS demonstrates a near-perfect correlation with contamination levels and outperforms existing baselines. Additionally, we perform comprehensive ablation studies to analyze the impact of key design choices, providing deeper insights into the components and effectiveness of KDS. These ablations highlight the importance of leveraging fine-grained kernel-based information and confirm the reliability of the proposed framework across diverse datasets and settings. Code is released in https://github.com/deeplearning-wisc/kernel-divergence-score.