mlsanityData to 7 Oct 2026

Survey: Leakage and Privacy at Inference Time

arXiv 2107.01614 ↗First posted 4 Jul 2021cs.LGcited by 44

Not on the map. Why?

Abstract (arXiv)

Leakage of data from publicly available Machine Learning (ML) models is an area of growing significance as commercial and government applications of ML can draw on multiple sources of data, potentially including users' and clients' sensitive data. We provide a comprehensive survey of contemporary advances on several fronts, covering involuntary data leakage which is natural to ML models, potential malevolent leakage which is caused by privacy attacks, and currently available defence mechanisms. We focus on inference-time leakage, as the most likely scenario for publicly available models. We first discuss what leakage is in the context of different data, tasks, and model architectures. We then propose a taxonomy across involuntary and malevolent leakage, available defences, followed by the currently available assessment metrics and applications. We conclude with outstanding challenges and open questions, outlining some promising directions for future research.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 134 of its 243 references that are in the map. First posted on arXiv 4 Jul 2021.

Builds on

134 of 243 references.

References · 134

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  8. 2016Deep Learning with Differential Privacycited by 2,682
  9. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  10. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  11. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  12. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
38 more
  1. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  2. 2017Deep Setscited by 1,394
  3. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  4. 2019Machine Unlearningcited by 1,020
  5. 2019Deep Leakage from Gradientscited by 1,004
  6. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  7. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  8. 2018How To Backdoor Federated Learningcited by 745
  9. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  10. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 660
  11. 2017Renyi Differential Privacycited by 643
  12. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  13. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  14. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  15. 2019Certified Data Removal from Machine Learning Modelscited by 512
  16. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  17. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  18. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  19. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
  20. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  21. 2020Label-Only Membership Inference Attackscited by 371
  22. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  23. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  24. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  25. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  26. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  27. 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 285
  28. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 273
  29. 2018Scalable Private Learning with PATEcited by 266
  30. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  31. 2018Adversarial Attacks and Defences: A Surveycited by 259
  32. 2020Approximate Data Deletion from Machine Learning Modelscited by 250
  33. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 242
  34. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 241
  35. 2018Differentially Private Generative Adversarial Networkcited by 235
  36. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
  37. 2019Adversarial Attacks and Defenses in Images, Graphs and Text: A Reviewcited by 234
  38. 2019High Accuracy and High Fidelity Extraction of Neural Networkscited by 208

50 most cited of 134 All 134 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 9; 2023: 9; 2024: 10; 2025: 10; 2026: 3.

44 papers.

Sort:

By area:

  1. 2022Reconstructing Training Data from Trained Neural Networkscited by 129
  2. 2023Machine Unlearning: Solutions and Challengescited by 85
  3. 2024Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligencecited by 65
  4. 2022A Customized Text Sanitization Mechanism with Differential Privacycited by 51
  5. 2021Learning Disentangled Representations in the Imaging Domaincited by 46
  6. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 44
  7. 2021Formalizing and Estimating Distribution Inference Riskscited by 42
  8. 2022A Survey on Gradient Inversion: Attacks, Defenses and Future Directionscited by 39
  9. 2024Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defensescited by 30
  10. 2023FedDiff: Diffusion Model Driven Federated Learning for Multi-Modal and Multi-Clientscited by 23
  11. 2022Subject Membership Inference Attacks in Federated Learningcited by 18
  12. 2023Privacy Distillation: Reducing Re-identification Risk of Multimodal Diffusion Modelscited by 18
32 more
  1. 2023Towards more Practical Threat Models in Artificial Intelligence Securitycited by 17
  2. 2023FRAMU: Attention-based Machine Unlearning using Federated Reinforcement Learningcited by 12
  3. 2024Is My Data in Your AI? Membership Inference Test (MINT) applied to Face Biometricscited by 12
  4. 2023FedFusion: Manifold Driven Federated Learning for Multi-satellite and Multi-modality Fusioncited by 8
  5. 2022Measuring Unintended Memorisation of Unique Private Features in Neural Networkscited by 5
  6. 2025Energy-Latency Attacks: A New Adversarial Threat to Deep Learningcited by 5
  7. 2025CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of Synthetic Chest Radiographscited by 5
  8. 2022GRAIMATTER Green Paper: Recommendations for disclosure control of trained Machine Learning (ML) models from Trusted Research Environments (TREs)cited by 4
  9. 2022Why patient data cannot be easily forgotten?cited by 3
  10. 2024MemControl: Mitigating Memorization in Diffusion Models via Automated Parameter Selectioncited by 3
  11. 2025A General Pseudonymization Framework for Cloud-Based LLMs: Replacing Privacy Information in Controlled Text Generationcited by 3
  12. 2024Differentially Private Integrated Decision Gradients (IDG-DP) for Radar-based Human Activity Recognitioncited by 2
  13. 2025A hierarchical approach for assessing the vulnerability of tree-based classification models to membership inference attackcited by 2
  14. 2025Engineering the Law-Machine Learning Translation Problem: Developing Legally Aligned Modelscited by 2
  15. 2025Privacy Risks and Preservation Methods in Explainable Artificial Intelligence: A Scoping Reviewcited by 2
  16. 2025Are Neuro-Inspired Multi-Modal Vision-Language Models Resilient to Membership Inference Privacy Leakage?cited by 2
  17. 2021Machine Learning Models Disclosure from Trusted Research Environments (TRE), Challenges and Opportunitiescited by 1
  18. 2022Privacy-aware Early Detection of COVID-19 through Adversarial Trainingcited by 1
  19. 2022Unintended memorisation of unique features in neural networkscited by 1
  20. 2023Privacy Protectability: An Information-theoretical Approachcited by 1
  21. 2026Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspectivecited by 1
  22. 2023Threats, Vulnerabilities, and Controls of Machine Learning Based Systems: A Survey and Taxonomycited by 0
  23. 2024A Quantization-based Technique for Privacy Preserving Distributed Learningcited by 0
  24. 2024A Different Level Text Protection Mechanism With Differential Privacycited by 0
  25. 2024Efficient Privacy-Preserving KAN Inference Using Homomorphic Encryptioncited by 0
  26. 2024Optimal Defenses Against Gradient Reconstruction Attackscited by 0
  27. 2024Inference Privacy: Properties and Mechanismscited by 0
  28. 2025Benchmarking Federated Machine Unlearning methods for Tabular Datacited by 0
  29. 2025Differentially Private 2D Human Pose Estimationcited by 0
  30. 2025Beyond Memorization: Selective Learning for Copyright-Safe Diffusion Model Trainingcited by 0
  31. 2026PriMod4AI: Lifecycle-Aware Privacy Threat Modeling for AI Systems using LLMcited by 0
  32. 2026MeGU: Machine-Guided Unlearning with Target Feature Disentanglementcited by 0