Perfectly Accurate Membership Inference by a Dishonest Central Server in Federated Learning
On the map: LLM safety and security → Federated unlearning and poisoning
Abstract (arXiv)
Federated Learning is expected to provide strong privacy guarantees, as only gradients or model parameters but no plain text training data is ever exchanged either between the clients or between the clients and the central server. In this paper, we challenge this claim by introducing a simple but still very effective membership inference attack algorithm, which relies only on a single training step. In contrast to the popular honest-but-curious model, we investigate a framework with a dishonest central server. Our strategy is applicable to models with ReLU activations and uses the properties of this activation function to achieve perfect accuracy. Empirical evaluation on visual classification tasks with MNIST, CIFAR10, CIFAR100 and CelebA datasets show that our method provides perfect accuracy in identifying one sample in a training set with thousands of samples. Occasional failures of our method lead us to discover duplicate images in the CIFAR100 and CelebA datasets.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security. Builds on 25 of its 38 references that are in the map. First posted on arXiv 30 Mar 2022.
Builds on
25 of 38 references.
References · 25
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2014Deep Learning Face Attributes in the Wildcited by 3,679
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
- 2019Deep Leakage from Gradientscited by 1,004
- 2018How To Backdoor Federated Learningcited by 745
- 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
- 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
- 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
- 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
- 2020iDLG: Improved Deep Leakage from Gradientscited by 367
- 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 242
13 more
- 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
- 2017Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryptioncited by 174
- 2017Machine Learning Models that Remember Too Muchcited by 165
- 2017Revisiting Distributed Synchronous SGDcited by 142
- 2021When the Curious Abandon Honesty: Federated Learning Is Not Privatecited by 126
- 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 106
- 2022Fishing for User Data in Large-Batch Federated Learning via Gradient Magnificationcited by 72
- 2021Eluding Secure Aggregation in Federated Learning via Model Inconsistencycited by 66
- 2020A Framework for Evaluating Gradient Leakage Attacks in Federated Learningcited by 63
- 2019Do We Train on Test Data? Purging CIFAR of Near-Duplicatescited by 61
- 2021Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrixcited by 43
- 2021Source Inference Attacks in Federated Learningcited by 31
- 2020An Accuracy-Lossless Perturbation Method for Defending Privacy Attacks in Federated Learningcited by 18
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 1.
2 papers.