# Understanding Membership Inferences on Well-Generalized Learning Models

arXiv: https://arxiv.org/abs/1802.04889 · HTML: https://mlsanity.com/paper/1802.04889/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2018.

## Built on by (most cited first, 30 of 90)

- [Detecting Pretraining Data from Large Language Models](https://mlsanity.com/paper/2310.16789/) (2023)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Deduplicating Training Data Mitigates Privacy Risks in Language Models](https://mlsanity.com/paper/2202.06539/) (2022)
- [Membership Inference Attacks against Language Models via Neighbourhood Comparison](https://mlsanity.com/paper/2305.18462/) (2023)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [Dataset Distillation using Neural Feature Regression](https://mlsanity.com/paper/2206.00719/) (2022)
- [On the Importance of Difficulty Calibration in Membership Inference Attacks](https://mlsanity.com/paper/2111.08440/) (2021)
- [Reconstructing Training Data from Trained Neural Networks](https://mlsanity.com/paper/2206.07758/) (2022)
- [A Comprehensive Survey of Dataset Distillation](https://mlsanity.com/paper/2301.05603/) (2023)
- [Practical Blind Membership Inference Attack via Differential Comparisons](https://mlsanity.com/paper/2101.01341/) (2021)
- [DE-COP: Detecting Copyrighted Content in Language Models Training Data](https://mlsanity.com/paper/2402.09910/) (2024)
- [Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture](https://mlsanity.com/paper/2110.08324/) (2021)
- [Scalable Membership Inference Attacks via Quantile Regression](https://mlsanity.com/paper/2307.03694/) (2023)
- [SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)](https://mlsanity.com/paper/2406.17975/) (2024)
- [Label-Only Model Inversion Attacks via Boundary Repulsion](https://mlsanity.com/paper/2203.01925/) (2022)
- [Survey: Leakage and Privacy at Inference Time](https://mlsanity.com/paper/2107.01614/) (2021)
- [Membership Inference on Word Embedding and Beyond](https://mlsanity.com/paper/2106.11384/) (2021)
- [Black-box Membership Inference Attacks against Fine-tuned Diffusion Models](https://mlsanity.com/paper/2312.08207/) (2023)
- [Measuring Data Leakage in Machine-Learning Models with Fisher Information](https://mlsanity.com/paper/2102.11673/) (2021)
- [Students Parrot Their Teachers: Membership Inference on Model Distillation](https://mlsanity.com/paper/2303.03446/) (2023)
- [Membership Inference Attacks against Large Vision-Language Models](https://mlsanity.com/paper/2411.02902/) (2024)
- [Identifying and Mitigating Privacy Risks Stemming from Language Models: A Survey](https://mlsanity.com/paper/2310.01424/) (2023)
- [Membership Inference Attacks Against Vision-Language Models](https://mlsanity.com/paper/2501.18624/) (2025)
- [Deletion Inference, Reconstruction, and Compliance in Machine (Un)Learning](https://mlsanity.com/paper/2202.03460/) (2022)
- [On the (In)security of Peer-to-Peer Decentralized Machine Learning](https://mlsanity.com/paper/2205.08443/) (2022)
- [PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps](https://mlsanity.com/paper/2403.09562/) (2024)
- [FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation](https://mlsanity.com/paper/2310.00339/) (2023)
- [Gaussian Membership Inference Privacy](https://mlsanity.com/paper/2306.07273/) (2023)
- [Leveraging Adversarial Examples to Quantify Membership Information Leakage](https://mlsanity.com/paper/2203.09566/) (2022)
- [On the Privacy Properties of GAN-generated Samples](https://mlsanity.com/paper/2206.01349/) (2022)

## Abstract (arXiv)

Membership Inference Attack (MIA) determines the presence of a record in a machine learning model's training data by querying the model. Prior work has shown that the attack is feasible when the model is overfitted to its training data or when the adversary controls the training algorithm. However, when the model is not overfitted and the adversary does not control the training algorithm, the threat is not well understood. In this paper, we report a study that discovers overfitting to be a sufficient but not a necessary condition for an MIA to succeed. More specifically, we demonstrate that even a well-generalized model contains vulnerable instances subject to a new generalized MIA (GMIA). In GMIA, we use novel techniques for selecting vulnerable instances and detecting their subtle influences ignored by overfitting metrics. Specifically, we successfully identify individual records with high precision in real-world datasets by querying black-box machine learning models. Further we show that a vulnerable record can even be indirectly attacked by querying other related records and existing generalization techniques are found to be less effective in protecting the vulnerable instances. Our findings sharpen the understanding of the fundamental cause of the problem: the unique influences the training instance may have on the model.