mlsanityData to 7 Oct 2026

Blind Baselines Beat Membership Inference Attacks for Foundation Models

arXiv 2406.16201 ↗First posted 23 Jun 2024cs.CR cs.CL cs.LGcited by 65

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Membership inference (MI) attacks try to determine if a data sample was used to train a machine learning model. For foundation models trained on unknown Web data, MI attacks are often used to detect copyrighted training materials, measure test set contamination, or audit machine unlearning. Unfortunately, we find that evaluations of MI attacks for foundation models are flawed, because they sample members and non-members from different distributions. For 8 published MI evaluation datasets, we show that blind attacks -- that distinguish the member and non-member distributions without looking at any trained model -- outperform state-of-the-art MI attacks. Existing evaluations thus tell us nothing about membership leakage of a foundation model's training data.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 35 of its 47 references that are in the map. First posted on arXiv 23 Jun 2024.

Builds on

35 of 47 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  2. 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models23 Oct
  3. 2023Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Study29 Sep
  4. 2023Towards More Realistic Membership Inference Attacks on Large Diffusion Models22 Jun
  5. 2022Membership Inference Attacks Against Text-to-image Generation Models3 Oct
  6. 2021Membership Inference Attacks From First Principles7 Dec

Other references · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  10. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  11. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  12. 2020Extracting Training Data from Large Language Modelscited by 2,144
17 more
  1. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  2. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  3. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  4. 2019Machine Unlearningcited by 1,020
  5. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  6. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  7. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  8. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
  9. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
  10. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  11. 2023Proving Test Set Contamination in Black Box Language Modelscited by 227
  12. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 108
  13. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  14. 2024DE-COP: Detecting Copyrighted Content in Language Models Training Datacited by 72
  15. 2024Copyright Traps for Large Language Modelscited by 36
  16. 2023MoPe: Model Perturbation-based Privacy Attacks on Language Modelscited by 25
  17. 2020Identifying Statistical Bias in Dataset Replicationcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 32; 2026: 18.

65 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  2. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 54
  3. 2025How much do language models memorize?cited by 53
  4. 2024Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Datacited by 49
  5. 2024Membership Inference Attacks against Large Vision-Language Modelscited by 31
  6. 2025Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generationcited by 26
  7. 2025Privacy Auditing of Large Language Modelscited by 22
  8. 2024Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Modelscited by 21
  9. 2024Context-Aware Membership Inference Attacks against Pre-trained Large Language Modelscited by 20
  10. 2025Language Models May Verbatim Complete Text They Were Not Explicitly Trained Oncited by 20
  11. 2024Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data?cited by 15
  12. 2024Ward: Provable RAG Dataset Inference via LLM Watermarkscited by 15
38 more
  1. 2025Privacy Attacks on Image AutoRegressive Modelscited by 15
  2. 2025Position: Adversarial ML for LLMs Is Not Making Any Progresscited by 14
  3. 2024Detecting Training Data of Large Language Models via Expectation Maximizationcited by 13
  4. 2025How Contaminated Is Your Benchmark? Quantifying Dataset Leakage in Large Language Models with Kernel Divergencecited by 13
  5. 2025Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemblecited by 11
  6. 2025SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attackscited by 9
  7. 2024LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM statescited by 8
  8. 2025STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasingscited by 8
  9. 2025Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspectivecited by 8
  10. 2024Fine-tuning can Help Detect Pretraining Data from Large Language Modelscited by 7
  11. 2025Unlocking Post-hoc Dataset Inference with Synthetic Datacited by 7
  12. 2025LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Modelscited by 7
  13. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  14. 2024A Statistical and Multi-Perspective Revisiting of the Membership Inference Attack in Large Language Modelscited by 5
  15. 2025Tag&Tab: Pretraining Data Detection in Large Language Models Using Keyword-Based Membership Inference Attackcited by 5
  16. 2025The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Modelscited by 5
  17. 2026Membership Inference Attacks Against Fine-tuned Diffusion Language Modelscited by 5
  18. 2025Synthetic Data Can Mislead Evaluations: Membership Inference as Machine Text Detectioncited by 3
  19. 2025Beyond Public Access in LLM Pre-Training Datacited by 3
  20. 2025Hey, That's My Data! Token-Only Dataset Inference in Large Language Modelscited by 3
  21. 2025Neural Breadcrumbs: Membership Inference Attacks on LLMs Through Hidden State and Attention Pattern Analysiscited by 3
  22. 2025Membership Inference Attacks on Tokenizers of Large Language Modelscited by 3
  23. 2025Revisiting Data Auditing in Large Vision-Language Modelscited by 2
  24. 2025What Really is a Member? Discrediting Membership Inference via Poisoningcited by 2
  25. 2025Memories Retrieved from Many Paths: A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Modelscited by 2
  26. 2026Models That Know How Evaluations Are Designed Score Safercited by 2
  27. 2026CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Modelscited by 2
  28. 2024Real-World Benchmarks Make Membership Inference Attacks Fail on Diffusion Modelscited by 1
  29. 2024Inner-Probe: Discovering Copyright-related Data Generation in LLM Architecturecited by 1
  30. 2024Watermarking Training Data of Music Generation Modelscited by 1
  31. 2025DIS-CO: Discovering Copyrighted Content in VLMs Training Datacited by 1
  32. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  33. 2025(Token-Level) InfoRMIA: Stronger Membership Inference and Memorization Assessment for LLMscited by 1
  34. 2026Quantifying the Effect of Test Set Contamination on Generative Evaluationscited by 1
  35. 2026Turning Black Box into White Box: Dataset Distillation Leakscited by 1
  36. 2026Membership Inference Attacks against Large Audio Language Modelscited by 1
  37. 2024Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attributioncited by 0
  38. 2025RECAP: Reproducing Copyrighted Data from LLMs Training with an Agentic Pipelinecited by 0

50 most cited of 65 All 65 (JSON)