# Probing Language Models for Pre-training Data Detection

arXiv: https://arxiv.org/abs/2406.01333 · HTML: https://mlsanity.com/paper/2406.01333/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 3 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark](https://mlsanity.com/paper/2310.18018/) (2023)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Membership Inference Attacks against Language Models via Neighbourhood Comparison](https://mlsanity.com/paper/2305.18462/) (2023)
- [Proving Test Set Contamination in Black Box Language Models](https://mlsanity.com/paper/2310.17623/) (2023)
- [Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference](https://mlsanity.com/paper/1906.11798/) (2019)
- [Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks](https://mlsanity.com/paper/2203.03929/) (2022)
- [On the Importance of Difficulty Calibration in Membership Inference Attacks](https://mlsanity.com/paper/2111.08440/) (2021)
- [Revisiting Membership Inference Under Realistic Assumptions](https://mlsanity.com/paper/2005.10881/) (2020)

## Built on by (most cited first, 25 of 25)

- [SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)](https://mlsanity.com/paper/2406.17975/) (2024)
- [Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Method](https://mlsanity.com/paper/2409.14781/) (2024)
- [A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models](https://mlsanity.com/paper/2502.17516/) (2025)
- [Learning to Refuse: Towards Mitigating Privacy Risks in LLMs](https://mlsanity.com/paper/2407.10058/) (2024)
- [False Sense of Security: Why Probing-based Malicious Input Detection Fails to Generalize](https://mlsanity.com/paper/2509.03888/) (2025)
- [LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM states](https://mlsanity.com/paper/2411.19876/) (2024)
- [SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attacks](https://mlsanity.com/paper/2506.10424/) (2025)
- [On The Fragility of Benchmark Contamination Detection in Reasoning Models](https://mlsanity.com/paper/2510.02386/) (2025)
- [Language Models Can Predict Their Own Behavior](https://mlsanity.com/paper/2502.13329/) (2025)
- [REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Model](https://mlsanity.com/paper/2509.22518/) (2025)
- [UAQFact: Evaluating Factual Knowledge Utilization of LLMs on Unanswerable Questions](https://mlsanity.com/paper/2505.23461/) (2025)
- [Reasoning Path and Latent State Analysis for Multi-view Visual Spatial Reasoning: A Cognitive Science Perspective](https://mlsanity.com/paper/2512.02340/) (2025)
- [CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models](https://mlsanity.com/paper/2606.17464/) (2026)
- [Automatic Calibration for Membership Inference Attack on Large Language Models](https://mlsanity.com/paper/2505.03392/) (2025)
- [Demystifying the Privacy-Utility Trade-off in LLM Interactions](https://mlsanity.com/paper/2609.10992/) (2026)
- [RN-F: A Novel Approach for Mitigating Contaminated Data in Large Language Models](https://mlsanity.com/paper/2505.13249/) (2025)
- [Are AI Coders Snitches? An Empirical Study of Pretraining Data Detection on Code Large Language Models](https://mlsanity.com/paper/2507.17389/) (2025)
- [EL-MIA: Quantifying Membership Inference Risks of Sensitive Entities in LLMs](https://mlsanity.com/paper/2511.00192/) (2025)
- [As If We've Met Before: LLMs Exhibit Certainty in Recognizing Seen Files](https://mlsanity.com/paper/2511.15192/) (2025)
- [From Unfamiliar to Familiar: Detecting Pre-training Data via Gradient Deviations in Large Language Models](https://mlsanity.com/paper/2603.04828/) (2026)
- [Excess Separability: Nuisance-Controlled Residual-Stream Probing for Benchmark Contamination Detection](https://mlsanity.com/paper/2608.12652/) (2026)
- [Vision-language models know more about agriculture than they show and rubric-grounded verifications close the gap](https://mlsanity.com/paper/2609.09417/) (2026)
- [CIG-MIA: Context-Induced Information Gain Membership Inference Attacks against Retrieval-Augmented Generation](https://mlsanity.com/paper/2609.14649/) (2026)
- [Detecting Pretraining Data in Large Language Models from a Free-Energy Perspective](https://mlsanity.com/paper/2609.21888/) (2026)
- [WildTrace: Evaluating Training Data Detection in Realistic Large-Scale LLM Post-Training](https://mlsanity.com/paper/2610.16152/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have shown their impressive capabilities, while also raising concerns about the data contamination problems due to privacy issues and leakage of benchmark datasets in the pre-training phase. Therefore, it is vital to detect the contamination by checking whether an LLM has been pre-trained on the target texts. Recent studies focus on the generated texts and compute perplexities, which are superficial features and not reliable. In this study, we propose to utilize the probing technique for pre-training data detection by examining the model's internal activations. Our method is simple and effective and leads to more trustworthy pre-training data detection. Additionally, we propose ArxivMIA, a new challenging benchmark comprising arxiv abstracts from Computer Science and Mathematics categories. Our experiments demonstrate that our method outperforms all baselines, and achieves state-of-the-art performance on both WikiMIA and ArxivMIA, with additional experiments confirming its efficacy (Our code and dataset are available at https://github.com/zhliu0106/probing-lm-data).