# Parameters or Privacy: A Provable Tradeoff Between Overparameterization and Membership Inference

arXiv: https://arxiv.org/abs/2202.01243 · HTML: https://mlsanity.com/paper/2202.01243/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 2 Feb 2022.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 26 of 26)

- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [Two models of double descent for weak features](https://mlsanity.com/paper/1903.07571/) (2019)
- [Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference](https://mlsanity.com/paper/1906.11798/) (2019)
- [Benign overfitting in ridge regression](https://mlsanity.com/paper/2009.14286/) (2020)
- [Fingerprinting Codes and the Price of Approximate Differential Privacy](https://mlsanity.com/paper/1311.3158/) (2013)
- [Practical Blind Membership Inference Attack via Differential Comparisons](https://mlsanity.com/paper/2101.01341/) (2021)
- [A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning](https://mlsanity.com/paper/2109.02355/) (2021)
- [Against Membership Inference Attack: Pruning is All You Need](https://mlsanity.com/paper/2008.13578/) (2020)
- [M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining](https://mlsanity.com/paper/2110.03888/) (2021)
- [Towards Measuring Membership Privacy](https://mlsanity.com/paper/1712.09136/) (2017)
- [On the Effectiveness of Regularization Against Membership Inference Attacks](https://mlsanity.com/paper/2006.05336/) (2020)
- [Distributed linear regression by averaging](https://mlsanity.com/paper/1810.00412/) (2018)
- [Accuracy-Privacy Trade-off in Deep Ensemble: A Membership Inference Perspective](https://mlsanity.com/paper/2105.05381/) (2021)
- [The Influence of Dropout on Membership Inference in Differentially Private Models](https://mlsanity.com/paper/2103.09008/) (2021)
- [Membership Inference Attacks on Lottery Ticket Networks](https://mlsanity.com/paper/2108.03506/) (2021)

## Built on by (most cited first, 14 of 14)

- [Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queries](https://mlsanity.com/paper/2210.10750/) (2022)
- [Gaussian Membership Inference Privacy](https://mlsanity.com/paper/2306.07273/) (2023)
- [A Blessing of Dimensionality in Membership Inference through Regularization](https://mlsanity.com/paper/2205.14055/) (2022)
- [Boomerang: Local sampling on image manifolds using diffusion models](https://mlsanity.com/paper/2210.12100/) (2022)
- [Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks](https://mlsanity.com/paper/2310.20579/) (2023)
- [Do Parameters Reveal More than Loss for Membership Inference?](https://mlsanity.com/paper/2406.11544/) (2024)
- [Federated Representation Learning in the Under-Parameterized Regime](https://mlsanity.com/paper/2406.04596/) (2024)
- [Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss](https://mlsanity.com/paper/2402.05453/) (2024)
- [Trustworthy AI: Safety, Bias, and Privacy -- A Survey](https://mlsanity.com/paper/2502.10450/) (2025)
- [Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning](https://mlsanity.com/paper/2410.06814/) (2024)
- [A hierarchical approach for assessing the vulnerability of tree-based classification models to membership inference attack](https://mlsanity.com/paper/2502.09396/) (2025)
- [Membership Privacy Risks of Sharpness Aware Minimization](https://mlsanity.com/paper/2310.00488/) (2023)
- [How Does Overparameterization Affect Machine Unlearning of Deep Neural Networks?](https://mlsanity.com/paper/2503.08633/) (2025)
- [Assessing Per-Sample Membership Inference Vulnerability without Retraining](https://mlsanity.com/paper/2602.15919/) (2026)

## Abstract (arXiv)

A surprising phenomenon in modern machine learning is the ability of a highly overparameterized model to generalize well (small error on the test data) even when it is trained to memorize the training data (zero error on the training data). This has led to an arms race towards increasingly overparameterized models (c.f., deep learning). In this paper, we study an underexplored hidden cost of overparameterization: the fact that overparameterized models may be more vulnerable to privacy attacks, in particular the membership inference attack that predicts the (potentially sensitive) examples used to train a model. We significantly extend the relatively few empirical results on this problem by theoretically proving for an overparameterized linear regression model in the Gaussian data setting that membership inference vulnerability increases with the number of parameters. Moreover, a range of empirical studies indicates that more complex, nonlinear models exhibit the same behavior. Finally, we extend our analysis towards ridge-regularized linear regression and show in the Gaussian data setting that increased regularization also increases membership inference vulnerability in the overparameterized regime.