mlsanityData to 14 Oct 2026

LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Construction

arXiv 2312.12343 ↗First posted 19 Dec 2023cs.CL cs.AIcited by 58

Not on the map. Why?

Abstract (arXiv)

Data contamination in evaluation is getting increasingly prevalent with the emergence of language models pre-trained on super large, automatically crawled corpora. This problem leads to significant challenges in the accurate assessment of model capabilities and generalisations. In this paper, we propose LatestEval, an automatic method that leverages the most recent texts to create uncontaminated reading comprehension evaluations. LatestEval avoids data contamination by only using texts published within a recent time window, ensuring no overlap with the training corpora of pre-trained language models. We develop the LatestEval automated pipeline to 1) gather the latest texts; 2) identify key information, and 3) construct questions targeting the information while removing the existing answers from the context. This encourages models to infer the answers themselves based on the remaining context, rather than just copy-paste. Our experiments demonstrate that language models exhibit negligible memorisation behaviours on LatestEval as opposed to previous benchmarks, suggesting a significantly reduced risk of data contamination and leading to a more robust evaluation. Data and code are publicly available at: https://github.com/liyucheng09/LatestEval.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

22 of 30 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2016Pointer Sentinel Mixture Modelscited by 3,096
  11. 2020Extracting Training Data from Large Language Modelscited by 2,189
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
10 more
  1. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  2. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  3. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  4. 2018QuAC : Question Answering in Contextcited by 381
  5. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  6. 2023Evaluating Verifiability in Generative Search Enginescited by 303
  7. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
  8. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 142
  9. 2023Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filteringcited by 65
  10. 2023Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model Evaluationcited by 48

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 17; 2025: 23; 2026: 18.

58 papers.

Sort:

By area:

  1. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 147
  2. 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 122
  3. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 82
  4. 2024Auto-Arena: Automating LLM Evaluations with Agent Peer Battles and Committee Discussionscited by 42
  5. 2024MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmarkcited by 41
  6. 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
  7. 2025A Survey on Data Contamination for Large Language Modelscited by 35
  8. 2024Evading Data Contamination Detection for Language Models is (too) Easycited by 30
  9. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
  10. 2024Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediationcited by 30
  11. 2025LLM Probability Concentration: How Alignment Shrinks the Generative Horizoncited by 26
  12. 2024UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMscited by 24
38 more
  1. 2024Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluationcited by 19
  2. 2025The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contaminationcited by 15
  3. 2024How Much Can We Forget about Data Contamination?cited by 14
  4. 2024Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdoutscited by 14
  5. 2025LLM as a Broken Telephone: Iterative Generation Distorts Informationcited by 12
  6. 2024DICE: Detecting In-distribution Contamination in LLM's Fine-tuning Phase for Math Reasoningcited by 11
  7. 2025Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completenesscited by 11
  8. 2024The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generationcited by 9
  9. 2025Emergent LLM behaviors are observationally equivalent to data leakagecited by 9
  10. 2025NetArena: Dynamic Benchmarks for AI Agents in Network Automationcited by 9
  11. 2025Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decodingcited by 9
  12. 2024Evaluating Large Language Models for Generalization and Robustness via Data Compressioncited by 5
  13. 2026A Neuropsychologically Grounded Evaluation of LLM Cognitive Abilitiescited by 5
  14. 2025AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Differencecited by 4
  15. 2025Facts Fade Fast: Evaluating Memorization of Outdated Medical Knowledge in Large Language Modelscited by 4
  16. 2025Establishing Trustworthy LLM Evaluation via Shortcut Neuron Analysiscited by 3
  17. 2025CoreEval: Automatically Building Contamination-Resilient Datasets with Real-World Knowledge toward Reliable LLM Evaluationcited by 3
  18. 2026The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidencecited by 3
  19. 2025Scalable and Reliable Evaluation of AI Knowledge Retrieval Systems: RIKER and the Coherent Simulated Universecited by 3
  20. 2026Markovian Generation Chains in Large Language Modelscited by 3
  21. 2024C2LEVA: Toward Comprehensive and Contamination-Free Language Model Evaluationcited by 2
  22. 2025FinS-Pilot: A Benchmark for Online Financial RAG Systemcited by 2
  23. 2025DiFaR: Enhancing Multimodal Misinformation Detection with Diverse, Factual, and Relevant Rationalescited by 2
  24. 2025Dual-Space Smoothness for Robust and Balanced LLM Unlearningcited by 2
  25. 2025Language Models Learn Universal Representations of Numbers and Here's Why You Should Carecited by 2
  26. 2026Distinguishable Deletion: Unifying Knowledge Erasure and Refusal for Large Language Model Unlearningcited by 2
  27. 2026What Proves You Wrong: Benchmarking Language Models on Falsifiable Research Ideationcited by 2
  28. 2024Agentic AI: The Era of Semantic Decodingcited by 1
  29. 2025AdEval: Alignment-based Dynamic Evaluation to Mitigate Data Contamination in Large Language Modelscited by 1
  30. 2025Improving Continual Pre-training Through Seamless Data Packingcited by 1
  31. 2025Provable Training Data Identification for Large Language Modelscited by 1
  32. 2025Evaluating Implicit Biases in LLM Reasoning through Logic Grid Puzzlescited by 1
  33. 2026MADE: A Living Benchmark for Multi-Label Text Classification with Uncertainty Quantification of Medical Device Adverse Eventscited by 1
  34. 2026TRACER: A Semantic-Aware Framework for Fine-Grained Contamination Detection in Code LLMscited by 1
  35. 2026Pretraining Data Exposure in Large Language Models: A Survey of Membership Inference, Data Contamination, and Security Implicationscited by 1
  36. 2026To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editingcited by 1
  37. 2025MINT: Multi-Vector Search Index Tuningcited by 0
  38. 2026FinForge: Semi-Synthetic Financial Benchmark Generationcited by 0

50 most cited of 58 All 58 (JSON)