mlsanityData to 14 Oct 2026

Bounding Information Leakage in Machine Learning

arXiv 2105.03875 ↗First posted 9 May 2021cs.LG cs.CR stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

Recently, it has been shown that Machine Learning models can leak sensitive information about their training data. This information leakage is exposed through membership and attribute inference attacks. Although many attack strategies have been proposed, little effort has been made to formalize these problems. We present a novel formalism, generalizing membership and attribute inference attack setups previously studied in the literature and connecting them to memorization and generalization. First, we derive a universal bound on the success rate of inference attacks and connect it to the generalization gap of the target model. Second, we study the question of how much sensitive information is stored by the algorithm about its training set and we derive bounds on the mutual information between the sensitive attributes and model parameters. Experimentally, we illustrate the potential of our approach by applying it to both synthetic data and classification tasks on natural images. Finally, we apply our formalism to different attribute inference strategies, with which an adversary is able to recover the identity of writers in the PenDigits dataset.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 29 of its 65 references that are in the map. First posted on arXiv 9 May 2021.

Builds on

29 of 65 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2016Deep Learning with Differential Privacycited by 2,728
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  5. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  6. 2016Stealing Machine Learning Models via Prediction APIscited by 619
  7. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 564
  8. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 520
  9. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  10. 2016DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Networkcited by 358
  11. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 334
  12. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 292
17 more
  1. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 271
  2. 2019MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examplescited by 197
  3. 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 175
  4. 2017Machine Learning Models that Remember Too Muchcited by 169
  5. 2019Privacy Risks of Securing Machine Learning Models against Adversarial Examplescited by 141
  6. 2018An overview of deep learning in medical imaging focusing on MRIcited by 140
  7. 2019Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learningcited by 122
  8. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 112
  9. 2020Machine Unlearning: Linear Filtration for Logit-based Classifierscited by 95
  10. 2018Understanding Membership Inferences on Well-Generalized Learning Modelscited by 90
  11. 2020An Overview of Privacy in Machine Learningcited by 39
  12. 2021On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Modelscited by 31
  13. 2020A Scalable Approach for Privacy-Preserving Collaborative Machine Learningcited by 21
  14. 2018Query-Efficient Black-Box Attack by Active Learningcited by 15
  15. 2020Not one but many Tradeoffs: Privacy Vs. Utility in Differentially Private Machine Learningcited by 13
  16. 2019Membership Model Inversion Attacks for Deep Networkscited by 4
  17. 2019Bayesian Differential Privacy for Machine Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 0; 2023: 3; 2024: 2; 2025: 3; 2026: 2.

11 papers.

  1. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 248
  2. 2023The Inadequacy of Similarity-based Privacy Metrics: Privacy Attacks against "Truly Anonymous" Synthetic Datasetscited by 14
  3. 2023Fundamental Limits of Membership Inference Attacks on Machine Learning Modelscited by 5
  4. 2024On the Impact of Uncertainty and Calibration on Likelihood-Ratio Membership Inference Attackscited by 4
  5. 2025Membership Inference Risks in Quantized Models: A Theoretical and Empirical Studycited by 3
  6. 2023Membership Privacy Risks of Sharpness Aware Minimizationcited by 2
  7. 2024Synthetic Data, Similarity-based Privacy Metrics, and Regulatory (Non-)Compliancecited by 1
  8. 2025PRIVET: PRoximIty leakage detection Via Extreme value Theorycited by 1
  9. 2026Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspectivecited by 1
  10. 2025Towards Reproducibility in Predictive Process Mining: SPICE -- A Deep Learning Librarycited by 0
  11. 2026Responsible AI for General-Purpose Systems: Overview, Challenges, and A Path Forwardcited by 0