# Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Method

arXiv: https://arxiv.org/abs/2409.14781 · HTML: https://mlsanity.com/paper/2409.14781/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Membership Inference Attacks against Language Models via Neighbourhood Comparison](https://mlsanity.com/paper/2305.18462/) (2023)
- [Proving Test Set Contamination in Black Box Language Models](https://mlsanity.com/paper/2310.17623/) (2023)
- [On the Importance of Difficulty Calibration in Membership Inference Attacks](https://mlsanity.com/paper/2111.08440/) (2021)
- [Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models](https://mlsanity.com/paper/2404.02936/) (2024)
- [Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities](https://mlsanity.com/paper/2308.12833/) (2023)
- [DE-COP: Detecting Copyrighted Content in Language Models Training Data](https://mlsanity.com/paper/2402.09910/) (2024)
- [Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?](https://mlsanity.com/paper/1904.05506/) (2019)
- [Membership Inference Attack Susceptibility of Clinical Language Models](https://mlsanity.com/paper/2104.08305/) (2021)
- [Improving Neural Response Diversity with Frequency-Aware Cross-Entropy Loss](https://mlsanity.com/paper/1902.09191/) (2019)
- [Probing Language Models for Pre-training Data Detection](https://mlsanity.com/paper/2406.01333/) (2024)
- [ChineseWebText: Large-scale High-quality Chinese Web Text Extracted with Effective Evaluation Model](https://mlsanity.com/paper/2311.01149/) (2023)
- [Concerned with Data Contamination? Assessing Countermeasures in Code Language Model](https://mlsanity.com/paper/2403.16898/) (2024)

## Built on by (most cited first, 30 of 55)

- [A Survey on Data Contamination for Large Language Models](https://mlsanity.com/paper/2502.14425/) (2025)
- [Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets](https://mlsanity.com/paper/2506.05346/) (2025)
- [Empirical Analysis of Decoding Biases in Masked Diffusion Models](https://mlsanity.com/paper/2508.13021/) (2025)
- [The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation](https://mlsanity.com/paper/2507.05578/) (2025)
- [Quantification of Large Language Model Distillation](https://mlsanity.com/paper/2501.12619/) (2025)
- [STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasings](https://mlsanity.com/paper/2504.13416/) (2025)
- [Window-based Membership Inference Attacks Against Fine-tuned Large Language Models](https://mlsanity.com/paper/2601.02751/) (2026)
- [Automated Detection of Pre-training Text in Black-box LLMs](https://mlsanity.com/paper/2506.19399/) (2025)
- [ICAS: Detecting Training Data from Autoregressive Image Generative Models](https://mlsanity.com/paper/2507.05068/) (2025)
- [Beyond Data Privacy: New Privacy Risks for Large Language Models](https://mlsanity.com/paper/2509.14278/) (2025)
- [Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models](https://mlsanity.com/paper/2510.09259/) (2025)
- [Generative AI collective behavior needs an interactionist paradigm](https://mlsanity.com/paper/2601.10567/) (2026)
- [Tag&Tab: Pretraining Data Detection in Large Language Models Using Keyword-Based Membership Inference Attack](https://mlsanity.com/paper/2501.08454/) (2025)
- [Self-Reflective Planning with Knowledge Graphs: Enhancing LLM Reasoning Reliability for Question Answering](https://mlsanity.com/paper/2505.19410/) (2025)
- [Hey, That's My Data! Token-Only Dataset Inference in Large Language Models](https://mlsanity.com/paper/2506.06057/) (2025)
- [Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs](https://mlsanity.com/paper/2509.13869/) (2025)
- [BudgetLeak: Membership Inference Attacks on RAG Systems via the Generation Budget Side Channel](https://mlsanity.com/paper/2511.12043/) (2025)
- [In-Context Probing for Membership Inference in Fine-Tuned Language Models](https://mlsanity.com/paper/2512.16292/) (2025)
- [Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework](https://mlsanity.com/paper/2601.03791/) (2026)
- [Membership Inference on LLMs in the Wild](https://mlsanity.com/paper/2601.11314/) (2026)
- [C2LEVA: Toward Comprehensive and Contamination-Free Language Model Evaluation](https://mlsanity.com/paper/2412.04947/) (2024)
- [Automatic Calibration for Membership Inference Attack on Large Language Models](https://mlsanity.com/paper/2505.03392/) (2025)
- [LogitTrace: Detecting Benchmark Contamination via Layerwise Logit Trajectories](https://mlsanity.com/paper/2509.20909/) (2025)
- [(Token-Level) InfoRMIA: Stronger Membership Inference and Memorization Assessment for LLMs](https://mlsanity.com/paper/2510.05582/) (2025)
- [Machine Text Detectors are Membership Inference Attacks](https://mlsanity.com/paper/2510.19492/) (2025)
- [Critical Confabulation: Can LLMs Hallucinate for Social Good?](https://mlsanity.com/paper/2511.07722/) (2025)
- [When Reasoning Leaks Membership: Membership Inference Attack on Black-box Large Reasoning Models](https://mlsanity.com/paper/2601.13607/) (2026)
- [DIS-CO: Discovering Copyrighted Content in VLMs Training Data](https://mlsanity.com/paper/2502.17358/) (2025)
- [Provable Training Data Identification for Large Language Models](https://mlsanity.com/paper/2510.09717/) (2025)
- [Forgetting-MarI: LLM Unlearning via Marginal Information Regularization](https://mlsanity.com/paper/2511.11914/) (2025)

## Abstract (arXiv)

As the scale of training corpora for large language models (LLMs) grows, model developers become increasingly reluctant to disclose details on their data. This lack of transparency poses challenges to scientific evaluation and ethical deployment. Recently, pretraining data detection approaches, which infer whether a given text was part of an LLM's training data through black-box access, have been explored. The Min-K% Prob method, which has achieved state-of-the-art results, assumes that a non-training example tends to contain a few outlier words with low token probabilities. However, the effectiveness may be limited as it tends to misclassify non-training texts that contain many common words with high probabilities predicted by LLMs. To address this issue, we introduce a divergence-based calibration method, inspired by the divergence-from-randomness concept, to calibrate token probabilities for pretraining data detection. We compute the cross-entropy (i.e., the divergence) between the token probability distribution and the token frequency distribution to derive a detection score. We have developed a Chinese-language benchmark, PatentMIA, to assess the performance of detection approaches for LLMs on Chinese text. Experimental results on English-language benchmarks and PatentMIA demonstrate that our proposed method significantly outperforms existing methods. Our code and PatentMIA benchmark are available at https://github.com/zhang-wei-chao/DC-PDD.