# Membership Inference Attacks on Large-Scale Models: A Survey

arXiv: https://arxiv.org/abs/2503.19338 · HTML: https://mlsanity.com/paper/2503.19338/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 43 of its 76 references that are in the map.
- First posted on arXiv 25 Mar 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)
- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 30 of 43)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Detecting Pretraining Data from Large Language Models](https://mlsanity.com/paper/2310.16789/) (2023)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [TrustLLM: Trustworthiness in Large Language Models](https://mlsanity.com/paper/2401.05561/) (2024)
- [Membership Inference Attacks against Language Models via Neighbourhood Comparison](https://mlsanity.com/paper/2305.18462/) (2023)
- [Do Membership Inference Attacks Work on Large Language Models?](https://mlsanity.com/paper/2402.07841/) (2024)
- [Security and Privacy Challenges of Large Language Models: A Survey](https://mlsanity.com/paper/2402.00888/) (2024)
- [Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks](https://mlsanity.com/paper/2203.03929/) (2022)
- [LLM Dataset Inference: Did you train on my dataset?](https://mlsanity.com/paper/2406.06443/) (2024)
- [Low-Cost High-Power Membership Inference Attacks](https://mlsanity.com/paper/2312.03262/) (2023)
- [SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)](https://mlsanity.com/paper/2406.17975/) (2024)
- [Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models](https://mlsanity.com/paper/2310.15007/) (2023)
- [Membership Inference Attack Susceptibility of Clinical Language Models](https://mlsanity.com/paper/2104.08305/) (2021)
- [Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generation](https://mlsanity.com/paper/2405.20446/) (2024)
- [Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Study](https://mlsanity.com/paper/2310.00108/) (2023)
- [M^4I: Multi-modal Models Membership Inference](https://mlsanity.com/paper/2209.06997/) (2022)
- [Membership Inference Attacks against Large Vision-Language Models](https://mlsanity.com/paper/2411.02902/) (2024)

## Built on by (most cited first, 18 of 18)

- [Vid-SME: Membership Inference Attacks against Large Video Understanding Models](https://mlsanity.com/paper/2506.03179/) (2025)
- [A Cost-Benefit Analysis of On-Premise Large Language Model Deployment: Breaking Even with Commercial LLM Services](https://mlsanity.com/paper/2509.18101/) (2025)
- [SMA: Who Said That? Auditing Membership Leakage in Semi-Black-box RAG Controlling](https://mlsanity.com/paper/2508.09105/) (2025)
- ['Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions](https://mlsanity.com/paper/2606.04906/) (2026)
- [VLALeaks: Membership Inference Attacks against Vision-Language-Action Models](https://mlsanity.com/paper/2606.15165/) (2026)
- [SoK: Challenges in Tabular Membership Inference Attacks](https://mlsanity.com/paper/2601.15874/) (2026)
- [Quantifying Memorization and Privacy Risks in Genomic Language Models](https://mlsanity.com/paper/2603.08913/) (2026)
- [Fast-MIA: Efficient and Scalable Membership Inference for LLMs](https://mlsanity.com/paper/2510.23074/) (2025)
- [Membership Inference Attack against Large Language Model-based Recommendation Systems: A New Distillation-based Paradigm](https://mlsanity.com/paper/2511.14763/) (2025)
- [PDR: A Plug-and-Play Positional Decay Framework for LLM Pre-training Data Detection](https://mlsanity.com/paper/2601.06827/) (2026)
- [AST-PAC: AST-guided Membership Inference for Code](https://mlsanity.com/paper/2602.13240/) (2026)
- [Sensitivity-Aware Retrieval-Augmented Intent Clarification](https://mlsanity.com/paper/2603.06025/) (2026)
- [Automated Membership Inference Attacks (AutoMIA): Discovering MIA Signal Computations using LLM Agents](https://mlsanity.com/paper/2603.19375/) (2026)
- [Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing](https://mlsanity.com/paper/2605.12876/) (2026)
- [AI Security Research Should Better Incentivize Defense Research](https://mlsanity.com/paper/2605.23448/) (2026)
- [LaRA: Layer-wise Representation Analysis for Detecting Data Contamination in RL Post-Training](https://mlsanity.com/paper/2605.29888/) (2026)
- [Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2](https://mlsanity.com/paper/2609.10935/) (2026)
- [SoK: Rethinking Membership Inference in Tabular Machine Learning](https://mlsanity.com/paper/2610.18486/) (2026)

## Abstract (arXiv)

As large-scale models such as Large Language Models (LLMs) and Large Multimodal Models (LMMs) see increasing deployment, their privacy risks remain underexplored. Membership Inference Attacks (MIAs), which reveal whether a data point was used in training the target model, are an important technique for exposing or assessing privacy risks and have been shown to be effective across diverse machine learning algorithms. However, despite extensive studies on MIAs in classic models, there remains a lack of systematic surveys addressing their effectiveness and limitations in large-scale models. To address this gap, we provide the first comprehensive review of MIAs targeting LLMs and LMMs, analyzing attacks by model type, adversarial knowledge, and strategy. Unlike prior surveys, we further examine MIAs across multiple stages of the model pipeline, including pre-training, fine-tuning, alignment, and Retrieval-Augmented Generation (RAG). Finally, we identify open challenges and propose future research directions for strengthening privacy resilience in large-scale models.