# Training on the Test Task Confounds Evaluation and Emergence

arXiv: https://arxiv.org/abs/2407.07890 · HTML: https://mlsanity.com/paper/2407.07890/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 71 of its 100 references that are in the map.
- First posted on arXiv 10 Jul 2024.

## Builds on (most cited first, 30 of 71)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)

## Built on by (most cited first, 30 of 34)

- [Generalizing Verifiable Instruction Following](https://mlsanity.com/paper/2507.02833/) (2025)
- [A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility](https://mlsanity.com/paper/2504.07086/) (2025)
- [Densing Law of LLMs](https://mlsanity.com/paper/2412.04315/) (2024)
- [A Rosetta Stone for AI Benchmarks](https://mlsanity.com/paper/2512.00193/) (2025)
- [YourBench: Easy Custom Evaluation Sets for Everyone](https://mlsanity.com/paper/2504.01833/) (2025)
- [Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models](https://mlsanity.com/paper/2412.02674/) (2024)
- [ONEBench to Test Them All: Sample-Level Benchmarking Over Open-Ended Capabilities](https://mlsanity.com/paper/2412.06745/) (2024)
- [The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination](https://mlsanity.com/paper/2503.16402/) (2025)
- [Instruction Following without Instruction Tuning](https://mlsanity.com/paper/2409.14254/) (2024)
- [Train-before-Test Harmonizes Language Model Rankings](https://mlsanity.com/paper/2507.05195/) (2025)
- [Leaving the barn door open for Clever Hans: Simple features predict LLM benchmark answers](https://mlsanity.com/paper/2410.11672/) (2024)
- [Scaling with Collapse: Efficient and Predictable Training of LLM Families](https://mlsanity.com/paper/2509.25087/) (2025)
- [On The Fragility of Benchmark Contamination Detection in Reasoning Models](https://mlsanity.com/paper/2510.02386/) (2025)
- [Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning](https://mlsanity.com/paper/2510.04786/) (2025)
- [Lawma: The Power of Specialization for Legal Annotation](https://mlsanity.com/paper/2407.16615/) (2024)
- [Position: Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead](https://mlsanity.com/paper/2507.23009/) (2025)
- [Train Once, Answer All: Many Pretraining Experiments for the Cost of One](https://mlsanity.com/paper/2509.23383/) (2025)
- [Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learning](https://mlsanity.com/paper/2506.10378/) (2025)
- [Data-Centric Lessons To Improve Speech-Language Pretraining](https://mlsanity.com/paper/2510.20860/) (2025)
- [Leaderboard Incentives: Model Rankings under Strategic Post-Training](https://mlsanity.com/paper/2603.08371/) (2026)
- [Evaluating the Robustness of Chinchilla Compute-Optimal Scaling](https://mlsanity.com/paper/2509.23963/) (2025)
- [Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs](https://mlsanity.com/paper/2509.25380/) (2025)
- [Prescriptive Scaling Reveals the Evolution of Language Model Capabilities](https://mlsanity.com/paper/2602.15327/) (2026)
- [Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspective](https://mlsanity.com/paper/2604.23267/) (2026)
- [How Hard is it to Rig a Benchmark? A Social Choice Analysis of Leaderboard Robustness](https://mlsanity.com/paper/2605.23628/) (2026)
- [Benchmark-Driven Selection of AI: Evidence from DeepSeek-R1](https://mlsanity.com/paper/2508.10173/) (2025)
- [Test-time RL alignment exposes task familiarity artifacts in LLM benchmarks](https://mlsanity.com/paper/2603.12875/) (2026)
- [Validity Threats for Foundation Model Research](https://mlsanity.com/paper/2606.05029/) (2026)
- [Are Multilingual Models Actually Improving? Isolating True Cross-Lingual Transfer](https://mlsanity.com/paper/2606.21954/) (2026)
- [Zero Gap Is Not Restoration: Stratified Per-Question Probability Evaluation and Step-wise Mitigation of Benchmark Contamination](https://mlsanity.com/paper/2608.07341/) (2026)

## Abstract (arXiv)

We study a fundamental problem in the evaluation of large language models that we call training on the test task. Unlike wrongful practices like training on the test data, leakage, or data contamination, training on the test task is not a malpractice. Rather, the term describes a growing set of practices that utilize knowledge about evaluation tasks at training time. We demonstrate that training on the test task confounds both relative model evaluations and claims about emergent capabilities. We argue that the seeming superiority of one model family over another may be explained by a different degree of training on the test task. To this end, we propose an effective method to adjust for the effect of training on the test task on benchmark evaluations. Put simply, to fine-tune each model under comparison on the same task-relevant data prior to evaluation. We then show that instances of emergent behavior disappear gradually as models train on the test task. Our work promotes a new perspective on the evaluation of large language models, with broad implications for benchmarking and the study of emergent capabilities.