mlsanityData to 14 Oct 2026

Membership Inference Attacks on Large-Scale Models: A Survey

arXiv 2503.19338 ↗First posted 25 Mar 2025cs.LG cs.CRcited by 18

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

As large-scale models such as Large Language Models (LLMs) and Large Multimodal Models (LMMs) see increasing deployment, their privacy risks remain underexplored. Membership Inference Attacks (MIAs), which reveal whether a data point was used in training the target model, are an important technique for exposing or assessing privacy risks and have been shown to be effective across diverse machine learning algorithms. However, despite extensive studies on MIAs in classic models, there remains a lack of systematic surveys addressing their effectiveness and limitations in large-scale models. To address this gap, we provide the first comprehensive review of MIAs targeting LLMs and LMMs, analyzing attacks by model type, adversarial knowledge, and strategy. Unlike prior surveys, we further examine MIAs across multiple stages of the model pipeline, including pre-training, fine-tuning, alignment, and Retrieval-Augmented Generation (RAG). Finally, we identify open challenges and propose future research directions for strengthening privacy resilience in large-scale models.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 43 of its 76 references that are in the map. First posted on arXiv 25 Mar 2025.

Builds on

43 of 76 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Membership Inference Attacks Against Vision-Language Models27 Jan
  2. 2024Membership Inference Attacks against Large Vision-Language Models5 Nov
  3. 2024Self-Comparison for Dataset-Level Membership Inference in Large (Vision-)Language Models16 Oct

Other references · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2020Learning to summarize from human feedbackcited by 2,588
  7. 2023A Survey of Large Language Modelscited by 2,576
  8. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  9. 2020Extracting Training Data from Large Language Modelscited by 2,189
  10. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  11. 2021Membership Inference Attacks From First Principlescited by 878
  12. 2023A Comprehensive Overview of Large Language Modelscited by 648
28 more
  1. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
  2. 2020Label-Only Membership Inference Attackscited by 388
  3. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  4. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 321
  5. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 285
  6. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 258
  7. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 195
  8. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  9. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 175
  10. 2024LLM Dataset Inference: Did you train on my dataset?cited by 110
  11. 2023Low-Cost High-Power Membership Inference Attackscited by 107
  12. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 59
  13. 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Modelscited by 56
  14. 2021Membership Inference Attack Susceptibility of Clinical Language Modelscited by 52
  15. 2024Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generationcited by 44
  16. 2023Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Studycited by 38
  17. 2022M^4I: Multi-modal Models Membership Inferencecited by 34
  18. 2024Mask-based Membership Inference Attacks for Retrieval-Augmented Generationcited by 29
  19. 2024Generating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generationcited by 25
  20. 2024Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignmentcited by 25
  21. 2024Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Modelscited by 24
  22. 2019Privacy Enhanced Multimodal Neural Representations for Emotion Recognitioncited by 22
  23. 2024Noisy Neighbors: Efficient membership inference attacks against LLMscited by 21
  24. 2024Semantic Membership Inference Attack against Large Language Modelscited by 17
  25. 2024Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directionscited by 14
  26. 2024LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM statescited by 10
  27. 2024Tracing Intricate Cues in Dialogue: Joint Graph Structure and Sentiment Dynamics for Multimodal Emotion Recognitioncited by 8
  28. 2024Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 13.

18 papers.

Sort:

By area:

  1. 2025Vid-SME: Membership Inference Attacks against Large Video Understanding Modelscited by 19
  2. 2025A Cost-Benefit Analysis of On-Premise Large Language Model Deployment: Breaking Even with Commercial LLM Servicescited by 17
  3. 2025SMA: Who Said That? Auditing Membership Leakage in Semi-Black-box RAG Controllingcited by 3
  4. 2026'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptionscited by 3
  5. 2026VLALeaks: Membership Inference Attacks against Vision-Language-Action Modelscited by 2
  6. 2026SoK: Challenges in Tabular Membership Inference Attackscited by 1
  7. 2026Quantifying Memorization and Privacy Risks in Genomic Language Modelscited by 1
  8. 2025Fast-MIA: Efficient and Scalable Membership Inference for LLMscited by 0
  9. 2025Membership Inference Attack against Large Language Model-based Recommendation Systems: A New Distillation-based Paradigmcited by 0
  10. 2026PDR: A Plug-and-Play Positional Decay Framework for LLM Pre-training Data Detectioncited by 0
  11. 2026AST-PAC: AST-guided Membership Inference for Codecited by 0
  12. 2026Sensitivity-Aware Retrieval-Augmented Intent Clarificationcited by 0
6 more
  1. 2026Automated Membership Inference Attacks (AutoMIA): Discovering MIA Signal Computations using LLM Agentscited by 0
  2. 2026Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothingcited by 0
  3. 2026AI Security Research Should Better Incentivize Defense Researchcited by 0
  4. 2026LaRA: Layer-wise Representation Analysis for Detecting Data Contamination in RL Post-Trainingcited by 0
  5. 2026Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2cited by 0
  6. 2026SoK: Rethinking Membership Inference in Tabular Machine Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models
  2. 2024Order of Magnitude Speedups for LLM Membership Inference
  3. 2024EM-MIAs: Enhancing Membership Inference Attacks in Large Language Models through Ensemble Modeling
  4. 2026What Hard Tokens Reveal: Exploiting Low-confidence Tokens for Membership Inference Attacks against Large Language Models
  5. 2025Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemble
  6. 2025On the Effectiveness of Membership Inference in Targeted Data Extraction from Large Language Models
  7. 2026A Full-Pipeline Framework for Evaluating Membership Inference Attacks in Machine Learning
  8. 2025Win-k: Improved Membership Inference Attacks on Small Language Models