Exposing the Vulnerability of Decentralized Learning to Membership Inference Attacks Through the Lens of Graph Mixing
On the map: LLM safety and security → Membership inference attacks
Abstract (arXiv)
The primary promise of decentralized learning is to allow users to engage in the training of machine learning models in a collaborative manner while keeping their data on their premises and without relying on any central entity. However, this paradigm necessitates the exchange of model parameters or gradients between peers. Such exchanges can be exploited to infer sensitive information about training data, which is achieved through privacy attacks (e.g., Membership Inference Attacks -- MIA). In order to devise effective defense mechanisms, it is important to understand the factors that increase/reduce the vulnerability of a given decentralized learning architecture to MIA. In this study, we extensively explore the vulnerability to MIA of various decentralized learning architectures by varying the graph structure (e.g., number of neighbors), the graph dynamics, and the aggregation strategy, across diverse datasets and data distributions. Our key finding, which to the best of our knowledge we are the first to report, is that the vulnerability to MIA is heavily correlated to (i) the local model mixing strategy performed by each node upon reception of models from neighboring nodes and (ii) the global mixing properties of the communication graph. We illustrate these results experimentally using four datasets and by theoretically analyzing the mixing properties of various decentralized architectures. We also empirically show that enhancing mixing properties is highly beneficial when combined with other privacy-preserving techniques such as Differential Privacy. Our paper draws a set of lessons learned for devising decentralized learning systems that reduce by design the vulnerability to MIA.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 43 of its 82 references that are in the map. First posted on arXiv 17 Dec 2024.
Builds on
43 of 82 references.
References · 43
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2021Model-Contrastive Federated Learningcited by 762
- 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
- 2018Federated Learning for Mobile Keyboard Predictioncited by 565
- 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
- 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
- 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
- 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
- 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
31 more
- 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
- 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 273
- 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
- 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 137
- 2020Byzantine-Resilient Secure Federated Learningcited by 93
- 2023Decentralized Federated Learning: A Survey and Perspectivecited by 86
- 2013Distributed optimization over time-varying directed graphscited by 77
- 2021Decentralized Federated Learning through Proxy Model Sharingcited by 56
- 2021Semi-Decentralized Federated Learning with Cooperative D2D Local Model Aggregationscited by 50
- 2020Decentralized Federated Learning via Mutual Knowledge Transfercited by 42
- 2021Federated Learning with Local Differential Privacy: Trade-offs between Privacy, Utility, and Communicationcited by 38
- 2024Evaluations of Machine Learning Privacy Defenses are Misleadingcited by 35
- 2022Communication-Efficient Topologies for Decentralized Learning with O(1) Consensus Ratecited by 25
- 2023Topology-aware Federated Learning in Edge Computing: A Comprehensive Surveycited by 23
- 2023Epidemic Learning: Boosting Decentralized Learning with Randomized Communicationcited by 21
- 2020An Accurate, Scalable and Verifiable Protocol for Federated Differentially Private Averagingcited by 20
- 2021Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?cited by 17
- 2024The Privacy Power of Correlated Noise in Decentralized Learningcited by 17
- 2018Accelerated Decentralized Optimization with Local Updates for Smooth and Strongly Convex Objectivescited by 11
- 2021Efficient and Reliable Overlay Networks for Decentralized Federated Learningcited by 9
- 2022Decentralized Event-Triggered Federated Learning with Heterogeneous Communication Thresholdscited by 9
- 2024Privacy Attacks in Decentralized Learningcited by 8
- 2024Differentially private and decentralized randomized power methodcited by 6
- 2021HADFL: Heterogeneity-aware Decentralized Federated Learning Frameworkcited by 5
- 2023Topology-Based Reconstruction Prevention for Decentralised Learningcited by 4
- 2022Efficient Fully Distributed Federated Learning with Adaptive Local Linkscited by 3
- 2024PeerSwap: A Peer-Sampler with Randomness Guaranteescited by 3
- 2016On Location Hiding in Distributed Systemscited by 1
- 2020On the Challenges of Detecting Side-Channel Attacks in SGXcited by 1
- 2023Evaluating the Impact of Local Differential Privacy on Utility Loss via Influence Functionscited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.