mlsanityData to 14 Oct 2026

Membership Inference on Word Embedding and Beyond

arXiv 2106.11384 ↗First posted 21 Jun 2021cs.CL cs.AI cs.CRcited by 35

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

In the text processing context, most ML models are built on word embeddings. These embeddings are themselves trained on some datasets, potentially containing sensitive data. In some cases this training is done independently, in other cases, it occurs as part of training a larger, task-specific model. In either case, it is of interest to consider membership inference attacks based on the embedding layer as a way of understanding sensitive information leakage. But, somewhat surprisingly, membership inference attacks on word embeddings and their effect in other natural language processing (NLP) tasks that use these embeddings, have remained relatively unexplored. In this work, we show that word embeddings are vulnerable to black-box membership inference attacks under realistic assumptions. Furthermore, we show that this leakage persists through two other major NLP applications: classification and text-generation, even when the embedding layer is not exposed to the attacker. We show that our MI attack achieves high attack accuracy against a classifier model and an LSTM-based language model. Indeed, our attack is a cheaper membership inference attack on text-generative models, which does not require the knowledge of the target model or any expensive training of text-generative models as shadow models.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 31 of its 47 references that are in the map. First posted on arXiv 21 Jun 2021.

Builds on

31 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Membership Inference Attacks on Machine Learning: A Survey14 Mar

Other references · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  4. 2016Deep Learning with Differential Privacycited by 2,728
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  6. 2020Extracting Training Data from Large Language Modelscited by 2,189
  7. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  8. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  9. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  10. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  11. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
  12. 2020Label-Only Membership Inference Attackscited by 388
18 more
  1. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  2. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 271
  3. 2020Information Leakage in Embedding Modelscited by 203
  4. 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 175
  5. 2020Membership Leakage in Label-Only Exposurescited by 168
  6. 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 104
  7. 2019Gmail Smart Compose: Real-Time Assisted Writingcited by 98
  8. 2018Understanding Membership Inferences on Well-Generalized Learning Modelscited by 90
  9. 2019Analyzing Information Leakage of Updates to Natural Language Modelscited by 90
  10. 2021Node-Level Membership Inference Attacks Against Graph Neural Networkscited by 84
  11. 2021Membership Inference Attack on Graph Neural Networkscited by 81
  12. 2019Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?cited by 71
  13. 2020Privacy in Deep Learning: A Surveycited by 71
  14. 2020ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learningcited by 70
  15. 2014word2vec Parameter Learning Explainedcited by 53
  16. 2018Towards Demystifying Membership Inference Attackscited by 40
  17. 2019Segmentations-Leak: Membership Inference Attacks and Defenses in Semantic Image Segmentationcited by 32
  18. 2021Privacy Regularization: Joint Privacy-Utility Optimization in Language Modelscited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 10; 2024: 14; 2025: 4; 2026: 2.

35 papers.

Sort:

By area:

  1. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  2. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  3. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  4. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 175
  5. 2024Investigating Data Contamination for Pre-training Language Modelscited by 91
  6. 2024Watermarking Makes Language Models Radioactivecited by 58
  7. 2022SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learningcited by 51
  8. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 51
  9. 2023Training Data Extraction From Pre-trained Language Models: A Surveycited by 45
  10. 2024Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generationcited by 44
  11. 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 36
  12. 2022M^4I: Multi-modal Models Membership Inferencecited by 34
23 more
  1. 2024Membership Inference Attacks against Large Vision-Language Modelscited by 33
  2. 2023Integration of Large Language Models and Federated Learningcited by 31
  3. 2023Identifying and Mitigating Privacy Risks Stemming from Language Models: A Surveycited by 30
  4. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 28
  5. 2024Data Contamination Calibration for Black-box LLMscited by 24
  6. 2022Subject Membership Inference Attacks in Federated Learningcited by 20
  7. 2023Stealing the Decoding Algorithms of Language Modelscited by 19
  8. 2025Vid-SME: Membership Inference Attacks against Large Video Understanding Modelscited by 19
  9. 2024AI Risk Management Should Incorporate Both Safety and Securitycited by 17
  10. 2024Towards Black-Box Membership Inference Attack for Diffusion Modelscited by 14
  11. 2024Detecting Training Data of Large Language Models via Expectation Maximizationcited by 13
  12. 2024Range Membership Inference Attackscited by 12
  13. 2024A Unified Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capabilitycited by 9
  14. 2025Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspectivecited by 9
  15. 2023Assessing Privacy Risks in Language Models: A Case Study on Summarization Taskscited by 8
  16. 2023Code Membership Inference for Detecting Unauthorized Data Use in Code Pre-trained Language Modelscited by 8
  17. 2023Improved Membership Inference Attacks Against Language Classification Modelscited by 7
  18. 2024PaCoST: Paired Confidence Significance Testing for Benchmark Contamination Detection in Large Language Modelscited by 6
  19. 2025Guess or Recall? Training CNNs to Classify and Localize Memorization in LLMscited by 1
  20. 2024LLMs Plagiarize: Ensuring Responsible Sourcing of Large Language Model Training Data Through Knowledge Graph Comparisoncited by 0
  21. 2025FiMMIA: scaling semantic perturbation-based membership inference across modalitiescited by 0
  22. 2026Automated Membership Inference Attacks (AutoMIA): Discovering MIA Signal Computations using LLM Agentscited by 0
  23. 2026Single-Sample Black-Box Membership Inference Attack against Vision-Language Models via Cross-modal Semantic Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Statistical and Multi-Perspective Revisiting of the Membership Inference Attack in Large Language Models
  2. 2025The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage
  3. 2025Membership Inference Attacks on Tokenizers of Large Language Models
  4. 2026Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2
  5. 2024Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models
  6. 2025Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models
  7. 2025The Hidden Cost of Modeling P(X): Vulnerability to Membership Inference Attacks in Generative Text Classifiers
  8. 2024Do Membership Inference Attacks Work on Large Language Models?