# Bounding Information Leakage in Machine Learning

arXiv: https://arxiv.org/abs/2105.03875 · HTML: https://mlsanity.com/paper/2105.03875/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 65 references that are in the map.
- First posted on arXiv 9 May 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network](https://mlsanity.com/paper/1606.00931/) (2016)
- [Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning](https://mlsanity.com/paper/1702.07464/) (2017)
- [Systematic Evaluation of Privacy Risks of Machine Learning Models](https://mlsanity.com/paper/2003.10595/) (2020)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples](https://mlsanity.com/paper/1909.10594/) (2019)
- [Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference](https://mlsanity.com/paper/1906.11798/) (2019)
- [Machine Learning Models that Remember Too Much](https://mlsanity.com/paper/1709.07886/) (2017)
- [Privacy Risks of Securing Machine Learning Models against Adversarial Examples](https://mlsanity.com/paper/1905.10291/) (2019)
- [An overview of deep learning in medical imaging focusing on MRI](https://mlsanity.com/paper/1811.10052/) (2018)
- [Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning](https://mlsanity.com/paper/1904.01067/) (2019)
- [Revisiting Membership Inference Under Realistic Assumptions](https://mlsanity.com/paper/2005.10881/) (2020)
- [Machine Unlearning: Linear Filtration for Logit-based Classifiers](https://mlsanity.com/paper/2002.02730/) (2020)
- [Understanding Membership Inferences on Well-Generalized Learning Models](https://mlsanity.com/paper/1802.04889/) (2018)
- [An Overview of Privacy in Machine Learning](https://mlsanity.com/paper/2005.08679/) (2020)
- [On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Models](https://mlsanity.com/paper/2103.07101/) (2021)
- [A Scalable Approach for Privacy-Preserving Collaborative Machine Learning](https://mlsanity.com/paper/2011.01963/) (2020)
- [Query-Efficient Black-Box Attack by Active Learning](https://mlsanity.com/paper/1809.04913/) (2018)
- [Not one but many Tradeoffs: Privacy Vs. Utility in Differentially Private Machine Learning](https://mlsanity.com/paper/2008.08807/) (2020)
- [Membership Model Inversion Attacks for Deep Networks](https://mlsanity.com/paper/1910.04257/) (2019)
- [Bayesian Differential Privacy for Machine Learning](https://mlsanity.com/paper/1901.09697/) (2019)

## Built on by (most cited first, 11 of 11)

- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [The Inadequacy of Similarity-based Privacy Metrics: Privacy Attacks against "Truly Anonymous" Synthetic Datasets](https://mlsanity.com/paper/2312.05114/) (2023)
- [Fundamental Limits of Membership Inference Attacks on Machine Learning Models](https://mlsanity.com/paper/2310.13786/) (2023)
- [On the Impact of Uncertainty and Calibration on Likelihood-Ratio Membership Inference Attacks](https://mlsanity.com/paper/2402.10686/) (2024)
- [Membership Inference Risks in Quantized Models: A Theoretical and Empirical Study](https://mlsanity.com/paper/2502.06567/) (2025)
- [Membership Privacy Risks of Sharpness Aware Minimization](https://mlsanity.com/paper/2310.00488/) (2023)
- [Synthetic Data, Similarity-based Privacy Metrics, and Regulatory (Non-)Compliance](https://mlsanity.com/paper/2407.16929/) (2024)
- [PRIVET: PRoximIty leakage detection Via Extreme value Theory](https://mlsanity.com/paper/2510.24233/) (2025)
- [Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective](https://mlsanity.com/paper/2601.22434/) (2026)
- [Towards Reproducibility in Predictive Process Mining: SPICE -- A Deep Learning Library](https://mlsanity.com/paper/2512.16715/) (2025)
- [Responsible AI for General-Purpose Systems: Overview, Challenges, and A Path Forward](https://mlsanity.com/paper/2601.13122/) (2026)

## Abstract (arXiv)

Recently, it has been shown that Machine Learning models can leak sensitive information about their training data. This information leakage is exposed through membership and attribute inference attacks. Although many attack strategies have been proposed, little effort has been made to formalize these problems. We present a novel formalism, generalizing membership and attribute inference attack setups previously studied in the literature and connecting them to memorization and generalization. First, we derive a universal bound on the success rate of inference attacks and connect it to the generalization gap of the target model. Second, we study the question of how much sensitive information is stored by the algorithm about its training set and we derive bounds on the mutual information between the sensitive attributes and model parameters. Experimentally, we illustrate the potential of our approach by applying it to both synthetic data and classification tasks on natural images. Finally, we apply our formalism to different attribute inference strategies, with which an adversary is able to recover the identity of writers in the PenDigits dataset.