mlsanityData to 7 Oct 2026

Detecting Benchmark Contamination Through Watermarking

arXiv 2502.17259 ↗First posted 24 Feb 2025cs.CR cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Benchmark contamination poses a significant challenge to the reliability of Large Language Models (LLMs) evaluations, as it is difficult to assert whether a model has been trained on a test set. We introduce a solution to this problem by watermarking benchmarks before their release. The embedding involves reformulating the original questions with a watermarked LLM, in a way that does not alter the benchmark utility. During evaluation, we can detect ``radioactivity'', ie traces that the text watermarks leave in the model during training, using a theoretically grounded statistical test. We test our method by pre-training 1B models from scratch on 10B tokens with controlled benchmark contamination, and validate its effectiveness in detecting contamination on ARC-Easy, ARC-Challenge, and MMLU. Results show similar benchmark utility post-watermarking and successful contamination detection when models are contaminated enough to enhance performance, eg p-val =10-3 for +5% on ARC-Easy.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 42 of its 44 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

42 of 44 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  12. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
30 more
  1. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  2. 2018Hierarchical Neural Story Generationcited by 1,182
  3. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  4. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  5. 2023A Watermark for Large Language Modelscited by 667
  6. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
  7. 2023Robust Distortion-free Watermarks for Language Modelscited by 279
  8. 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 227
  9. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  10. 2023Undetectable Watermarks for Language Modelscited by 208
  11. 2023Rethinking Benchmark and Contamination for Language Models with Rephrased Samplescited by 192
  12. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 188
  13. 2024A Careful Examination of Large Language Model Performance on Grade School Arithmeticcited by 188
  14. 2023On the Reliability of Watermarks for Large Language Modelscited by 170
  15. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  16. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  17. 2023Who Wrote this Code? Watermarking for Code Generationcited by 116
  18. 2023A Semantic Invariant Robust Watermark for Large Language Modelscited by 97
  19. 2024Investigating Data Contamination for Pre-training Language Modelscited by 91
  20. 2023Protecting Language Generation Models via Invisible Watermarkingcited by 81
  21. 2023Three Bricks to Consolidate Watermarks for Large Language Modelscited by 80
  22. 2023On the Learnability of Watermarks for Language Modelscited by 72
  23. 2020Radioactive data: tracing through trainingcited by 67
  24. 2024Adaptive Text Watermark for Large Language Modelscited by 60
  25. 2024Watermarking Makes Language Models Radioactivecited by 52
  26. 2023Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Modelscited by 46
  27. 2023Watermarking Conditional Text Generation for AI Detection: Unveiling Challenges and a Semantic-Aware Watermark Remedycited by 45
  28. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 36
  29. 2024Evaluation data contamination in LLMs: how do we measure it and (when) does it matter?cited by 30
  30. 2024Ward: Provable RAG Dataset Inference via LLM Watermarkscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasingscited by 8
  2. 2026Antidistillation Fingerprintingcited by 6
  3. 2025How Good is Post-Hoc Watermarking With Language Model Rephrasing?cited by 2
  4. 2026TextSeal: A Localized LLM Watermark for Provenance & Distillation Protectioncited by 2
  5. 2026Dataset Watermarking with Provable Black-Box Detectioncited by 0
  6. 2026Watermarking for Proprietary Dataset Protectioncited by 0