# An Overview of Privacy in Machine Learning

arXiv: https://arxiv.org/abs/2005.08679 · HTML: https://mlsanity.com/paper/2005.08679/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2020.

## Built on by (most cited first, 30 of 39)

- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Trustworthy AI: A Computational Perspective](https://mlsanity.com/paper/2107.06641/) (2021)
- [ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models](https://mlsanity.com/paper/2102.02551/) (2021)
- [Survey: Leakage and Privacy at Inference Time](https://mlsanity.com/paper/2107.01614/) (2021)
- [Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity](https://mlsanity.com/paper/2401.07348/) (2024)
- [Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs](https://mlsanity.com/paper/2105.12049/) (2021)
- [Membership Inference Attacks Against Self-supervised Speech Models](https://mlsanity.com/paper/2111.05113/) (2021)
- [Bounding Information Leakage in Machine Learning](https://mlsanity.com/paper/2105.03875/) (2021)
- [Learning from Aggregated Data: Curated Bags versus Random Bags](https://mlsanity.com/paper/2305.09557/) (2023)
- [Trained Random Forests Completely Reveal your Dataset](https://mlsanity.com/paper/2402.19232/) (2024)
- [Exploiting Fairness to Enhance Sensitive Attributes Reconstruction](https://mlsanity.com/paper/2209.01215/) (2022)
- [Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources](https://mlsanity.com/paper/2305.06217/) (2023)
- [PRICURE: Privacy-Preserving Collaborative Inference in a Multi-Party Setting](https://mlsanity.com/paper/2102.09751/) (2021)
- [On the Alignment of Group Fairness with Attribute Privacy](https://mlsanity.com/paper/2211.10209/) (2022)
- [Can Querying for Bias Leak Protected Attributes? Achieving Privacy With Smooth Sensitivity](https://mlsanity.com/paper/2211.02139/) (2022)
- [FaceMAE: Privacy-Preserving Face Recognition via Masked Autoencoders](https://mlsanity.com/paper/2205.11090/) (2022)
- [Probabilistic Dataset Reconstruction from Interpretable Models](https://mlsanity.com/paper/2308.15099/) (2023)
- [Model Reporting for Certifiable AI: A Proposal from Merging EU Regulation into AI Development](https://mlsanity.com/paper/2307.11525/) (2023)
- [SoK: Unintended Interactions among Machine Learning Defenses and Risks](https://mlsanity.com/paper/2312.04542/) (2023)
- [SoK: Can Trajectory Generation Combine Privacy and Utility?](https://mlsanity.com/paper/2403.07218/) (2024)
- [MixNN: Protection of Federated Learning Against Inference Attacks by Mixing Neural Network Layers](https://mlsanity.com/paper/2109.12550/) (2021)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [Are Neuromorphic Architectures Inherently Privacy-preserving? An Exploratory Study](https://mlsanity.com/paper/2411.06613/) (2024)
- [Label-only Model Inversion Attack: The Attack that Requires the Least Information](https://mlsanity.com/paper/2203.06555/) (2022)
- [Privacy Measurement in Tabular Synthetic Data: State of the Art and Future Research Directions](https://mlsanity.com/paper/2311.17453/) (2023)
- [SoK: Taming the Triangle -- On the Interplays between Fairness, Interpretability and Privacy in Machine Learning](https://mlsanity.com/paper/2312.16191/) (2023)
- [From Counterfactuals to Trees: Competitive Analysis of Model Extraction Attacks](https://mlsanity.com/paper/2502.05325/) (2025)
- [When Better Features Mean Greater Risks: The Performance-Privacy Trade-Off in Contrastive Learning](https://mlsanity.com/paper/2506.05743/) (2025)
- [Holistic risk assessment of inference attacks in machine learning](https://mlsanity.com/paper/2212.10628/) (2022)
- [Learning from Aggregate responses: Instance Level versus Bag Level Loss Functions](https://mlsanity.com/paper/2401.11081/) (2024)

## Abstract (arXiv)

Over the past few years, providers such as Google, Microsoft, and Amazon have started to provide customers with access to software interfaces allowing them to easily embed machine learning tasks into their applications. Overall, organizations can now use Machine Learning as a Service (MLaaS) engines to outsource complex tasks, e.g., training classifiers, performing predictions, clustering, etc. They can also let others query models trained on their data. Naturally, this approach can also be used (and is often advocated) in other contexts, including government collaborations, citizen science projects, and business-to-business partnerships. However, if malicious users were able to recover data used to train these models, the resulting information leakage would create serious issues. Likewise, if the inner parameters of the model are considered proprietary information, then access to the model should not allow an adversary to learn such parameters. In this document, we set to review privacy challenges in this space, providing a systematic review of the relevant research literature, also exploring possible countermeasures. More specifically, we provide ample background information on relevant concepts around machine learning and privacy. Then, we discuss possible adversarial models and settings, cover a wide range of attacks that relate to private and/or sensitive information leakage, and review recent results attempting to defend against such attacks. Finally, we conclude with a list of open problems that require more work, including the need for better evaluations, more targeted defenses, and the study of the relation to policy and data protection efforts.