mlsanityData to 14 Oct 2026

Data Contamination Calibration for Black-box LLMs

arXiv 2405.11930 ↗First posted 20 May 2024cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

The rapid advancements of Large Language Models (LLMs) tightly associate with the expansion of the training data size. However, the unchecked ultra-large-scale training sets introduce a series of potential risks like data contamination, i.e. the benchmark data is used for training. In this work, we propose a holistic method named Polarized Augment Calibration (PAC) along with a new to-be-released dataset to detect the contaminated data and diminish the contamination effect. PAC extends the popular MIA (Membership Inference Attack) -- from machine learning community -- by forming a more global target at detecting training data to Clarify invisible training data. As a pioneering work, PAC is very much plug-and-play that can be integrated with most (if not all) current white- and black-box LLMs. By extensive experiments, PAC outperforms existing methods by at least 4.5%, towards data contamination detection on more 4 dataset formats, with more than 10 base LLMs. Besides, our application in real-world scenarios highlights the prominent presence of contamination and related issues.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 42 of its 61 references that are in the map. First posted on arXiv 20 May 2024.

Builds on

42 of 61 references.

References · 42

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2020Scaling Laws for Neural Language Modelscited by 6,586
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  11. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  12. 2020Extracting Training Data from Large Language Modelscited by 2,189
30 more
  1. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  2. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  3. 2021Membership Inference Attacks From First Principlescited by 878
  4. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  5. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  6. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
  7. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  8. 2019Release Strategies and the Social Impacts of Language Modelscited by 513
  9. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  10. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  11. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  12. 2020Label-Only Membership Inference Attackscited by 388
  13. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 258
  14. 2022Data Contamination: From Memorization to Exploitationcited by 236
  15. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 199
  16. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  17. 2018Machine Learning with Membership Privacy using Adversarial Regularizationcited by 186
  18. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 175
  19. 2023Platypus: Quick, Cheap, and Powerful Refinement of LLMscited by 150
  20. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 147
  21. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  22. 2023Privacy Auditing with One (1) Training Runcited by 123
  23. 2019Analyzing Information Leakage of Updates to Natural Language Modelscited by 90
  24. 2019Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAScited by 83
  25. 2018Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learningcited by 77
  26. 2023Training Data Extraction From Pre-trained Language Models: A Surveycited by 45
  27. 2023"According to ...": Prompting Language Models Improves Quoting from Pre-Training Datacited by 42
  28. 2020A Dataset and Baselines for Visual Question Answering on Artcited by 38
  29. 2021Membership Inference on Word Embedding and Beyondcited by 35
  30. 2023Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibilitycited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 12; 2026: 7.

24 papers.

Sort:

By area:

  1. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 59
  2. 2025A Survey on Data Contamination for Large Language Modelscited by 35
  3. 2024Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediationcited by 30
  4. 2025How Contaminated Is Your Benchmark? Quantifying Dataset Leakage in Large Language Models with Kernel Divergencecited by 15
  5. 2025The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contaminationcited by 15
  6. 2025DCR: Quantifying Data Contamination in LLMs Evaluationcited by 10
  7. 2024Fine-tuning can Help Detect Pretraining Data from Large Language Modelscited by 8
  8. 2025Table as a Modality for Large Language Modelscited by 7
  9. 2024A Statistical and Multi-Perspective Revisiting of the Membership Inference Attack in Large Language Modelscited by 6
  10. 2025Automated Detection of Pre-training Text in Black-box LLMscited by 6
  11. 2025AcademicEval: Live Long-Context LLM Benchmarkcited by 4
  12. 2025LogitTrace: Detecting Benchmark Contamination via Layerwise Logit Trajectoriescited by 2
12 more
  1. 2026CoLA: A Choice Leakage Attack Framework to Expose Privacy Risks in Subset Trainingcited by 2
  2. 2025AdEval: Alignment-based Dynamic Evaluation to Mitigate Data Contamination in Large Language Modelscited by 1
  3. 2026Obscuring Data Contamination Through Translation: Evidence from Arabic Corporacited by 1
  4. 2024TDDBench: A Benchmark for Training data detectioncited by 0
  5. 2025Fast-MIA: Efficient and Scalable Membership Inference for LLMscited by 0
  6. 2025EL-MIA: Quantifying Membership Inference Risks of Sensitive Entities in LLMscited by 0
  7. 2025REMIND: Input Loss Landscapes Reveal Residual Memorization in Post-Unlearning LLMscited by 0
  8. 2026What Hard Tokens Reveal: Exploiting Low-confidence Tokens for Membership Inference Attacks against Large Language Modelscited by 0
  9. 2026AST-PAC: AST-guided Membership Inference for Codecited by 0
  10. 2026Combating Data Laundering in LLM Trainingcited by 0
  11. 2026Calibrating Post-Training Feature Shifts for LLM Data Contamination Detectioncited by 0
  12. 2026Detecting Pretraining Data in Large Language Models from a Free-Energy Perspectivecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptions
  2. 2024Both Text and Images Leaked! A Systematic Analysis of Data Contamination in Multimodal LLM
  3. 2024Clean Evaluations on Contaminated Visual Language Models
  4. 2023Rethinking Benchmark and Contamination for Language Models with Rephrased Samples
  5. 2023Task Contamination: Language Models May Not Be Few-Shot Anymore
  6. 2026Uncertainty-based Debiasing and Unlearning for Decontamination
  7. 2025Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models
  8. 2026Detecting Data Contamination in Large Language Models