mlsanityData to 7 Oct 2026

Probing Language Models for Pre-training Data Detection

arXiv 2406.01333 ↗First posted 3 Jun 2024cs.CL cs.AIcited by 24

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have shown their impressive capabilities, while also raising concerns about the data contamination problems due to privacy issues and leakage of benchmark datasets in the pre-training phase. Therefore, it is vital to detect the contamination by checking whether an LLM has been pre-trained on the target texts. Recent studies focus on the generated texts and compute perplexities, which are superficial features and not reliable. In this study, we propose to utilize the probing technique for pre-training data detection by examining the model's internal activations. Our method is simple and effective and leads to more trustworthy pre-training data detection. Additionally, we propose ArxivMIA, a new challenging benchmark comprising arxiv abstracts from Computer Science and Mathematics categories. Our experiments demonstrate that our method outperforms all baselines, and achieves state-of-the-art performance on both WikiMIA and ArxivMIA, with additional experiments confirming its efficacy (Our code and dataset are available at https://github.com/zhliu0106/probing-lm-data).

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 24 of its 33 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

24 of 33 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  8. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  9. 2020Extracting Training Data from Large Language Modelscited by 2,144
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  11. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  12. 2016Understanding intermediate layers using linear classifier probescited by 1,279
12 more
  1. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  2. 2021Membership Inference Attacks From First Principlescited by 839
  3. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  4. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  5. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  6. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  7. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  8. 2023Proving Test Set Contamination in Black Box Language Modelscited by 227
  9. 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 168
  10. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  11. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 137
  12. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 108

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 13; 2026: 7.

24 papers.

Sort:

By area:

  1. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 54
  2. 2024Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Methodcited by 54
  3. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  4. 2024Learning to Refuse: Towards Mitigating Privacy Risks in LLMscited by 22
  5. 2025False Sense of Security: Why Probing-based Malicious Input Detection Fails to Generalizecited by 15
  6. 2025SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attackscited by 9
  7. 2024LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM statescited by 8
  8. 2025Language Models Can Predict Their Own Behaviorcited by 8
  9. 2025On The Fragility of Benchmark Contamination Detection in Reasoning Modelscited by 8
  10. 2025REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Modelcited by 4
  11. 2025UAQFact: Evaluating Factual Knowledge Utilization of LLMs on Unanswerable Questionscited by 3
  12. 2025Automatic Calibration for Membership Inference Attack on Large Language Modelscited by 2
12 more
  1. 2025Reasoning Path and Latent State Analysis for Multi-view Visual Spatial Reasoning: A Cognitive Science Perspectivecited by 2
  2. 2026CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Modelscited by 2
  3. 2026Demystifying the Privacy-Utility Trade-off in LLM Interactionscited by 1
  4. 2025RN-F: A Novel Approach for Mitigating Contaminated Data in Large Language Modelscited by 0
  5. 2025Are AI Coders Snitches? An Empirical Study of Pretraining Data Detection on Code Large Language Modelscited by 0
  6. 2025EL-MIA: Quantifying Membership Inference Risks of Sensitive Entities in LLMscited by 0
  7. 2025As If We've Met Before: LLMs Exhibit Certainty in Recognizing Seen Filescited by 0
  8. 2026From Unfamiliar to Familiar: Detecting Pre-training Data via Gradient Deviations in Large Language Modelscited by 0
  9. 2026Excess Separability: Nuisance-Controlled Residual-Stream Probing for Benchmark Contamination Detectioncited by 0
  10. 2026Vision-language models know more about agriculture than they show and rubric-grounded verifications close the gapcited by 0
  11. 2026CIG-MIA: Context-Induced Information Gain Membership Inference Attacks against Retrieval-Augmented Generationcited by 0
  12. 2026Detecting Pretraining Data in Large Language Models from a Free-Energy Perspectivecited by 0