# Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning

arXiv: https://arxiv.org/abs/1812.00910 · HTML: https://mlsanity.com/paper/1812.00910/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2018.

## Built on by (most cited first, 30 of 77)

- [The Privacy Onion Effect: Memorization is Relative](https://mlsanity.com/paper/2206.10469/) (2022)
- [Property Inference From Poisoning](https://mlsanity.com/paper/2101.11073/) (2021)
- [PUMA: Performance Unchanged Model Augmentation for Training Data Removal](https://mlsanity.com/paper/2203.00846/) (2022)
- [Gradient-Leakage Resilient Federated Learning](https://mlsanity.com/paper/2107.01154/) (2021)
- [Copyright Traps for Large Language Models](https://mlsanity.com/paper/2402.09363/) (2024)
- [FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data](https://mlsanity.com/paper/2103.03918/) (2021)
- [On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Models](https://mlsanity.com/paper/2103.07101/) (2021)
- [Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics](https://mlsanity.com/paper/2401.05146/) (2024)
- [Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment](https://mlsanity.com/paper/2407.06443/) (2024)
- [Data Contamination Calibration for Black-box LLMs](https://mlsanity.com/paper/2405.11930/) (2024)
- [On the (In)security of Peer-to-Peer Decentralized Machine Learning](https://mlsanity.com/paper/2205.08443/) (2022)
- [Membership Inference Attack Using Self Influence Functions](https://mlsanity.com/paper/2205.13680/) (2022)
- [Gradient Leakage Attack Resilient Deep Learning](https://mlsanity.com/paper/2112.13178/) (2021)
- [GPFL: Simultaneously Learning Global and Personalized Feature Information for Personalized Federated Learning](https://mlsanity.com/paper/2308.10279/) (2023)
- [DReS-FL: Dropout-Resilient Secure Federated Learning for Non-IID Clients via Secret Data Sharing](https://mlsanity.com/paper/2210.02680/) (2022)
- [Subspace based Federated Unlearning](https://mlsanity.com/paper/2302.12448/) (2023)
- [Differential Private Federated Transfer Learning for Mental Health Monitoring in Everyday Settings: A Case Study on Stress Detection](https://mlsanity.com/paper/2402.10862/) (2024)
- [Towards Causal Federated Learning For Enhanced Robustness and Privacy](https://mlsanity.com/paper/2104.06557/) (2021)
- [A Review of the Role of Causality in Developing Trustworthy AI Systems](https://mlsanity.com/paper/2302.06975/) (2023)
- [Securing Distributed SGD against Gradient Leakage Threats](https://mlsanity.com/paper/2305.06473/) (2023)
- [Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach](https://mlsanity.com/paper/2311.14632/) (2023)
- [Trustworthy Distributed AI Systems: Robustness, Privacy, and Governance](https://mlsanity.com/paper/2402.01096/) (2024)
- [DPIS: An Enhanced Mechanism for Differentially Private SGD with Importance Sampling](https://mlsanity.com/paper/2210.09634/) (2022)
- [Accuracy-Privacy Trade-off in Deep Ensemble: A Membership Inference Perspective](https://mlsanity.com/paper/2105.05381/) (2021)
- [Do Parameters Reveal More than Loss for Membership Inference?](https://mlsanity.com/paper/2406.11544/) (2024)
- [Personalized Semi-Supervised Federated Learning for Human Activity Recognition](https://mlsanity.com/paper/2104.08094/) (2021)
- [Meta Federated Learning](https://mlsanity.com/paper/2102.05561/) (2021)
- [Similarity Distribution based Membership Inference Attack on Person Re-identification](https://mlsanity.com/paper/2211.15918/) (2022)
- [Skellam Mixture Mechanism: a Novel Approach to Federated Learning with Differential Privacy](https://mlsanity.com/paper/2212.04371/) (2022)
- [A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective](https://mlsanity.com/paper/2402.03688/) (2024)

## Abstract (arXiv)

Deep neural networks are susceptible to various inference attacks as they remember information about their training data. We design white-box inference attacks to perform a comprehensive privacy analysis of deep learning models. We measure the privacy leakage through parameters of fully trained models as well as the parameter updates of models during training. We design inference algorithms for both centralized and federated learning, with respect to passive and active inference attackers, and assuming different adversary prior knowledge. We evaluate our novel white-box membership inference attacks against deep learning algorithms to trace their training data records. We show that a straightforward extension of the known black-box attacks to the white-box setting (through analyzing the outputs of activation functions) is ineffective. We therefore design new algorithms tailored to the white-box setting by exploiting the privacy vulnerabilities of the stochastic gradient descent algorithm, which is the algorithm used to train deep neural networks. We investigate the reasons why deep learning models may leak information about their training data. We then show that even well-generalized models are significantly susceptible to white-box membership inference attacks, by analyzing state-of-the-art pre-trained and publicly available models for the CIFAR dataset. We also show how adversarial participants, in the federated learning setting, can successfully run active membership inference attacks against other participants, even when the global model achieves high prediction accuracies.