mlsanityData to 7 Oct 2026

Pretraining Data Exposure in Large Language Models: A Survey of Membership Inference, Data Contamination, and Security Implications

arXiv 2605.26133 ↗First posted 21 May 2026cs.CL cs.AI cs.LGcited by 1

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Large Language Models (LLMs) have become the predominant paradigm in NLP, advancing both research and industry. As model sizes and pretraining data grow, concerns about Pretraining Data Exposure (PDE) increase due to the scale and opacity of training datasets. PDE refers to determining whether specific data appeared in an LLM's pretraining corpus. It is critical for ensuring evaluation integrity and protecting privacy, intersecting two key areas: data contamination and membership inference. Though conceptually related, these areas have often been studied in isolation. This paper offers the first unified survey of both under the PDE framework. We formalize PDE across exposure levels, review attack and defense methods, synthesize empirical findings, and highlight open challenges and future research directions.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 57 of its 67 references that are in the map. First posted on arXiv 21 May 2026.

Builds on

57 of 67 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Is My Text in Your AI Model? Gradient-based Membership Inference Test applied to LLMs10 Mar
  2. 2025Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Training27 Feb
  3. 2024Detecting Training Data of Large Language Models via Expectation Maximization10 Oct
  4. 2024Order of Magnitude Speedups for LLM Membership Inference22 Sep
  5. 2024Semantic Membership Inference Attack against Large Language Models14 Jun
  6. 2024Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach22 Apr
  7. 2024Sampling-based Pseudo-Likelihood for Membership Inference Attacks17 Apr
  8. 2024Protecting Copyrighted Material with Unique Identifiers in Large Language Model Training23 Mar
  9. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  10. 2023Code Membership Inference for Detecting Unauthorized Data Use in Code Pre-trained Language Models12 Dec
  11. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration10 Nov
  12. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparison29 May
3 more
  1. 2023Analyzing Leakage of Personally Identifiable Information in Language Models1 Feb
  2. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks8 Mar
  3. 2021Membership Inference Attacks on Machine Learning: A Survey14 Mar

Other references · 42

  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  2. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  3. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  4. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  5. 2023Who's Harry Potter? Approximate Unlearning in LLMscited by 366
  6. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  7. 2023Proving Test Set Contamination in Black Box Language Modelscited by 227
  8. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  9. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 139
  10. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  11. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 136
  12. 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 119
30 more
  1. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 98
  2. 2024RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Modelscited by 77
  3. 2023LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Constructioncited by 57
  4. 2023Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model Evaluationcited by 47
  5. 2023Training Data Extraction From Pre-trained Language Models: A Surveycited by 43
  6. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 42
  7. 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
  8. 2024Copyright Traps for Large Language Modelscited by 36
  9. 2024ConStat: Performance-Based Contamination Detection in Large Language Modelscited by 33
  10. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspectivecited by 33
  11. 2025A Survey on Data Contamination for Large Language Modelscited by 33
  12. 2024Evading Data Contamination Detection for Language Models is (too) Easycited by 30
  13. 2024Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediationcited by 30
  14. 2024VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbationcited by 29
  15. 2024Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Modelscited by 28
  16. 2024Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptionscited by 26
  17. 2023Watermarking Text Data on Large Language Models for Dataset Copyrightcited by 24
  18. 2024TRUCE: Private Benchmarking to Prevent Contamination and Improve Comparative Evaluation of LLMscited by 24
  19. 2024Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluationcited by 18
  20. 2024Towards Data Contamination Detection for Modern Large Language Models: Limitations, Inconsistencies, and Oracle Challengescited by 18
  21. 2024Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data?cited by 15
  22. 2024Concerned with Data Contamination? Assessing Countermeasures in Code Language Modelcited by 14
  23. 2025Does Training with Synthetic Data Truly Protect Privacy?cited by 12
  24. 2023Gotcha! This Model Uses My Code! Evaluating Membership Leakage Risks in Code Modelscited by 11
  25. 2024Pandora's White-Box: Precise Training Data Detection and Extraction in Large Language Modelscited by 11
  26. 2024A Taxonomy for Data Contamination in Large Language Modelscited by 11
  27. 2024Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearningcited by 9
  28. 2024Trained Without My Consent: Detecting Code Inclusion In Language Models Trained on Codecited by 8
  29. 2024MemHunter: Automated and Verifiable Memorization Detection at Dataset-scale in LLMscited by 2
  30. 2025AdEval: Alignment-based Dynamic Evaluation to Mitigate Data Contamination in Large Language Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 6