Communication-Efficient Secure Aggregation in Decentralized Learning
Not on the map. Why?
Abstract (arXiv)
Decentralized learning (DL) enables participants to collaboratively train models without a central server, yet it faces significant scalability challenges that demand sparsification to reduce the prohibitive communication costs of peer-to-peer exchange. While secure aggregation effectively mitigates privacy risks in standard settings, it has remained fundamentally incompatible with sparsification in decentralized networks due to the mismatch of indices across local updates, forcing a trade-off between communication efficiency and privacy. This paper introduces CESAR, a novel protocol that resolves this incompatibility by integrating secure aggregation and sparsification to provide provable defense against honest-but-curious and colluding adversaries. By coordinating masks over parameter intersections, CESAR supports node dropouts and robust privacy without central aggregation. Empirical evaluations on models up to 124 million parameters demonstrate that CESAR matches the accuracy of non-private baselines while cutting total data exchange by 66.7 % compared to a standard full-parameter decentralized protocol (D-PSGD). With TopK sparsification on IID data, CESAR even exceeds by 0.3 % the accuracy achieved by D-PSGD with sparsification. Collectively, these results establish CESAR as the first decentralized protocol to achieve both privacy and communication efficiency through secure aggregation in DL.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 13 May 2024.
Builds on
23 of 41 references.
References · 23
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
- 2019Deep Leakage from Gradientscited by 1,004
- 2021Membership Inference Attacks From First Principlescited by 839
- 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
- 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
- 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
- 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 288
- 2018The Convergence of Sparsified Gradient Methodscited by 228
- 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
- 2018Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentationcited by 148
- 2020A Survey of Knowledge-Enhanced Text Generationcited by 119
11 more
- 2022Gradient Obfuscation Gives a False Sense of Security in Federated Learningcited by 37
- 2021On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Modelscited by 30
- 2022Muffliato: Peer-to-Peer Privacy Amplification for Decentralized Optimization and Averagingcited by 27
- 2022On the (In)security of Peer-to-Peer Decentralized Machine Learningcited by 23
- 2020Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selectioncited by 20
- 2023Decentralized Learning Made Easy with DecentralizePycited by 18
- 2023Flamingo: Multi-Round Single-Server Secure Aggregation with Applications to Private Federated Learningcited by 16
- 2024Noiseless Privacy-Preserving Decentralized Learningcited by 9
- 2021Sparsified Secure Aggregation for Privacy-Preserving Federated Learningcited by 8
- 2023Get More for Less in Decentralized Learning Systemscited by 6
- 2024Low-Cost Privacy-Preserving Decentralized Learningcited by 4
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.