mlsanityData to 7 Oct 2026

Detecting Training Data of Large Language Models via Expectation Maximization

arXiv 2410.07582 ↗First posted 10 Oct 2024cs.CL cs.AI cs.CRcited by 13

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Membership inference attacks (MIAs) aim to determine whether a specific example was used to train a given language model. While prior work has explored prompt-based attacks such as ReCALL, these methods rely heavily on the assumption that using known non-members as prompts reliably suppresses the model's responses to non-member queries. We propose EM-MIA, a new membership inference approach that iteratively refines prefix effectiveness and membership scores using an expectation-maximization strategy without requiring labeled non-member examples. To support controlled evaluation, we introduce OLMoMIA, a benchmark that enables analysis of MIA robustness under systematically varied distributional overlap and difficulty. Experiments on WikiMIA and OLMoMIA show that EM-MIA outperforms existing baselines, particularly in settings with clear distributional separability. We highlight scenarios where EM-MIA succeeds in practical settings with partial distributional overlap, while failure cases expose fundamental limitations of current MIA methods under near-identical conditions. We release our code and evaluation pipeline to encourage reproducible and robust MIA research.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 59 of its 77 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

59 of 77 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024Nob-MIAs: Non-biased Membership Inference Attacks Assessment on Large Language Models with Ex-Post Dataset Construction12 Aug
  2. 2024Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment8 Jul
  3. 2024Blind Baselines Beat Membership Inference Attacks for Foundation Models23 Jun
  4. 2024ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods23 Jun
  5. 2024Semantic Membership Inference Attack against Large Language Models14 Jun
  6. 2024LLM Dataset Inference: Did you train on my dataset?10 Jun
  7. 2024Sampling-based Pseudo-Likelihood for Membership Inference Attacks17 Apr
  8. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  9. 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models23 Oct
  10. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparison29 May
  11. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks8 Mar
  12. 2021Membership Inference Attacks From First Principles7 Dec
3 more
  1. 2021Enhanced Membership Inference Attacks against Machine Learning Models18 Nov
  2. 2021Membership Inference on Word Embedding and Beyond21 Jun
  3. 2021Membership Inference Attack Susceptibility of Clinical Language Models16 Apr

Other references · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  10. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  11. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  12. 2020Extracting Training Data from Large Language Modelscited by 2,144
32 more
  1. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  2. 2018Datasheets for Datasetscited by 1,371
  3. 2023Large Language Modelscited by 774
  4. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  5. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  6. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  7. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  8. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  9. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
  10. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
  11. 2023Scaling Data-Constrained Language Modelscited by 391
  12. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  13. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  14. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  15. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  16. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  17. 2022Data Contamination: From Memorization to Exploitationcited by 234
  18. 2023Proving Test Set Contamination in Black Box Language Modelscited by 227
  19. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  20. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 167
  21. 2023D4: Improving LLM Pretraining via Document De-Duplication and Diversificationcited by 146
  22. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 127
  23. 2023Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4cited by 123
  24. 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 119
  25. 2023Foundation Models and Fair Usecited by 112
  26. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  27. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  28. 2024DE-COP: Detecting Copyrighted Content in Language Models Training Datacited by 72
  29. 2023User Inference Attacks on Large Language Modelscited by 36
  30. 2024Copyright Traps for Large Language Modelscited by 36
  31. 2024Data Contamination Report from the 2024 CONDA Shared Taskcited by 19
  32. 2024DICE: Detecting In-distribution Contamination in LLM's Fine-tuning Phase for Math Reasoningcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 5.

13 papers.

Sort:

By area:

  1. 2025ReLearn: Unlearning via Learning for Large Language Modelscited by 20
  2. 2024LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM statescited by 8
  3. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  4. 2025In-Context Probing for Membership Inference in Fine-Tuned Language Modelscited by 3
  5. 2025Efficient Knowledge Graph Unlearning with Zeroth-order Informationcited by 2
  6. 2026CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Modelscited by 2
  7. 2025Black Box Absorption: LLMs Undermining Innovative Ideascited by 1
  8. 2026Toward Reliable, Safe, and Secure LLMs for Scientific Applicationscited by 1
  9. 2026Pretraining Data Exposure in Large Language Models: A Survey of Membership Inference, Data Contamination, and Security Implicationscited by 1
  10. 2025A Systematic Analysis of Base Model Choice for Reward Modelingcited by 0
  11. 2025GPT and Prejudice: A Sparse Approach to Understanding Learned Representations in Large Language Modelscited by 0
  12. 2026Detecting Data Contamination in Large Language Modelscited by 0
1 more
  1. 2026LLM Unlearning for Cyber Defense: A Survey on Methods, Challenges, and Emerging Threatscited by 0