# Fundamental Limits of Membership Inference Attacks on Machine Learning Models

arXiv: https://arxiv.org/abs/2310.13786 · HTML: https://mlsanity.com/paper/2310.13786/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 28 of its 49 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 28 of 28)

- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [Gradient Descent Maximizes the Margin of Homogeneous Neural Networks](https://mlsanity.com/paper/1906.05890/) (2019)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models](https://mlsanity.com/paper/2205.10770/) (2022)
- [Information Leakage in Embedding Models](https://mlsanity.com/paper/2004.00053/) (2020)
- [Directional convergence and alignment in deep learning](https://mlsanity.com/paper/2006.06657/) (2020)
- [Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference](https://mlsanity.com/paper/1906.11798/) (2019)
- [Machine Learning Models that Remember Too Much](https://mlsanity.com/paper/1709.07886/) (2017)
- [The Privacy Onion Effect: Memorization is Relative](https://mlsanity.com/paper/2206.10469/) (2022)
- [ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning](https://mlsanity.com/paper/2007.09339/) (2020)
- [Measuring Data Leakage in Machine-Learning Models with Fisher Information](https://mlsanity.com/paper/2102.11673/) (2021)
- [Gradient Methods Provably Converge to Non-Robust Networks](https://mlsanity.com/paper/2202.04347/) (2022)
- [Membership Inference Attacks on Deep Regression Models for Neuroimaging](https://mlsanity.com/paper/2105.02866/) (2021)
- [Membership Inference Attacks and Generalization: A Causal Perspective](https://mlsanity.com/paper/2209.08615/) (2022)
- [Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fano](https://mlsanity.com/paper/2210.13662/) (2022)
- [Membership-Doctor: Comprehensive Assessment of Membership Inference Against Machine Learning Models](https://mlsanity.com/paper/2208.10445/) (2022)
- [Bounding Membership Inference](https://mlsanity.com/paper/2202.12232/) (2022)
- [Bounding Information Leakage in Machine Learning](https://mlsanity.com/paper/2105.03875/) (2021)
- [Measuring Unintended Memorisation of Unique Private Features in Neural Networks](https://mlsanity.com/paper/2202.08099/) (2022)
- [Provable Membership Inference Privacy](https://mlsanity.com/paper/2211.06582/) (2022)

## Built on by (most cited first, 5 of 5)

- [On the Impact of Uncertainty and Calibration on Likelihood-Ratio Membership Inference Attacks](https://mlsanity.com/paper/2402.10686/) (2024)
- [Membership Inference Risks in Quantized Models: A Theoretical and Empirical Study](https://mlsanity.com/paper/2502.06567/) (2025)
- [A Critical Review on the Effectiveness and Privacy Threats of Membership Inference Attacks](https://mlsanity.com/paper/2603.22987/) (2026)
- [FairNVT: Fair Classification via Noise Injection in Vision Transformers](https://mlsanity.com/paper/2604.16780/) (2026)
- [Discard the Dross and Select the Essential: Pre-query Sample Selection for Black-box Membership Inference Attacks](https://mlsanity.com/paper/2606.30081/) (2026)

## Abstract (arXiv)

Membership inference attacks (MIA) can reveal whether a particular data point was part of the training dataset, potentially exposing sensitive information about individuals. This article provides theoretical guarantees by exploring the fundamental statistical limitations associated with MIAs on machine learning models at large. More precisely, we first derive the statistical quantity that governs the effectiveness and success of such attacks. We then theoretically prove that in a non-linear regression setting with overfitting learning procedures, attacks may have a high probability of success. Finally, we investigate several situations for which we provide bounds on this quantity of interest. Interestingly, our findings indicate that discretizing the data might enhance the learning procedure's security. Specifically, it is demonstrated to be limited by a constant, which quantifies the diversity of the underlying data distribution. We illustrate those results through simple simulations.