# Language model developers should report train-test overlap

arXiv: https://arxiv.org/abs/2410.08385 · HTML: https://mlsanity.com/paper/2410.08385/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 47 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Builds on (most cited first, 21 of 21)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Detecting Pretraining Data from Large Language Models](https://mlsanity.com/paper/2310.16789/) (2023)
- [Proving Test Set Contamination in Black Box Language Models](https://mlsanity.com/paper/2310.17623/) (2023)
- [Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets](https://mlsanity.com/paper/2008.02637/) (2020)
- [Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models](https://mlsanity.com/paper/2402.15938/) (2024)
- [Granite Code Models: A Family of Open Foundation Models for Code Intelligence](https://mlsanity.com/paper/2405.04324/) (2024)
- [Investigating Data Contamination for Pre-training Language Models](https://mlsanity.com/paper/2401.06059/) (2024)
- [Open-LLM-Leaderboard: From Multi-choice to Open-style Questions for LLMs Evaluation, Benchmark, and Arena](https://mlsanity.com/paper/2406.07545/) (2024)
- [Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluation](https://mlsanity.com/paper/2102.01818/) (2021)
- [The Foundation Model Transparency Index](https://mlsanity.com/paper/2310.12941/) (2023)
- [Ecosystem Graphs: The Social Footprint of Foundation Models](https://mlsanity.com/paper/2303.15772/) (2023)
- [Koala: An Index for Quantifying Overlaps with Pre-training Corpora](https://mlsanity.com/paper/2303.14770/) (2023)
- [A benchmark for toxic comment classification on Civil Comments dataset](https://mlsanity.com/paper/2301.11125/) (2023)

## Built on by (most cited first, 16 of 16)

- [Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation](https://mlsanity.com/paper/2502.06559/) (2025)
- [Crosslingual Reasoning through Test-Time Scaling](https://mlsanity.com/paper/2505.05408/) (2025)
- [Language Models May Verbatim Complete Text They Were Not Explicitly Trained On](https://mlsanity.com/paper/2503.17514/) (2025)
- [Frontier AI's Impact on the Cybersecurity Landscape](https://mlsanity.com/paper/2504.05408/) (2025)
- [WorldMedQA-V: a multilingual, multimodal medical examination dataset for multimodal language models evaluation](https://mlsanity.com/paper/2410.12722/) (2024)
- [The 2025 Foundation Model Transparency Index](https://mlsanity.com/paper/2512.10169/) (2025)
- [BenchAgents: Multi-Agent Systems for Structured Benchmark Creation](https://mlsanity.com/paper/2410.22584/) (2024)
- [How Do LLMs Fail In Agentic Scenarios? A Qualitative Analysis of Success and Failure Scenarios of Various LLMs in Agentic Simulations](https://mlsanity.com/paper/2512.07497/) (2025)
- [Position: Ensuring mutual privacy is necessary for effective external evaluation of proprietary AI systems](https://mlsanity.com/paper/2503.01470/) (2025)
- [Towards a Standard, Enterprise-Relevant Agentic AI Benchmark: Lessons from 5.5 billion tokens' worth of agentic AI evaluations](https://mlsanity.com/paper/2511.08042/) (2025)
- [AI Evaluation Should Require Standardized Item-Level Data Releases](https://mlsanity.com/paper/2604.03244/) (2026)
- [Is MT Ready for the Next Crisis or Pandemic?](https://mlsanity.com/paper/2601.10082/) (2026)
- [SetLexSem Challenge: Using Set Operations to Evaluate the Lexical and Semantic Robustness of Language Models](https://mlsanity.com/paper/2411.07336/) (2024)
- [A Conceptual Framework for AI Capability Evaluations](https://mlsanity.com/paper/2506.18213/) (2025)
- [The Limits of AI Data Transparency Policy: Three Disclosure Fallacies](https://mlsanity.com/paper/2601.18127/) (2026)
- [An Investigation of the NeurIPS and ICML 2025 Position Tracks](https://mlsanity.com/paper/2608.16894/) (2026)

## Abstract (arXiv)

Language models are extensively evaluated, but correctly interpreting evaluation results requires knowledge of train-test overlap which refers to the extent to which the language model is trained on the very data it is being tested on. The public currently lacks adequate information about train-test overlap: most models have no public train-test overlap statistics, and third parties cannot directly measure train-test overlap since they do not have access to the training data. To make this clear, we document the practices of 30 model developers, finding that just 9 developers report train-test overlap: 4 developers release training data under open-source licenses, enabling the community to directly measure train-test overlap, and 5 developers publish their train-test overlap methodology and statistics. By engaging with language model developers, we provide novel information about train-test overlap for three additional developers. Overall, we take the position that language model developers should publish train-test overlap statistics and/or training data whenever they report evaluation results on public test sets. We hope our work increases transparency into train-test overlap to increase the community-wide trust in model evaluations.