FedMPDD: Communication-Efficient Federated Learning with Privacy Preservation Attributes via Projected Directional Derivative
Not on the map. Why?
Abstract (arXiv)
This paper introduces textttFedMPDD (Federated Learning via Multi-Projected Directional Derivatives), a novel algorithm that simultaneously optimizes bandwidth utilization and enhances privacy in Federated Learning. The core idea of textttFedMPDD is to encode each client's high-dimensional gradient by computing its directional derivatives along multiple random vectors. This compresses the gradient into a much smaller message, significantly reducing uplink communication costs from O(d) to O(m), where m ll d. The server then decodes the aggregated information by projecting it back onto the same random vectors. Our key insight is that averaging multiple projections overcomes the dimension-dependent convergence limitations of a single projection. We provide a rigorous theoretical analysis, establishing that textttFedMPDD converges at a rate of O(1/√K), matching the performance of FedSGD. Furthermore, we demonstrate that our method provides some inherent privacy against gradient inversion attacks due to the geometric properties of low-rank projections, offering a tunable privacy-utility trade-off controlled by the number of projections. Extensive experiments on benchmark datasets validate our theory and demonstrates our results.
Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 51 of its 68 references that are in the map. First posted on arXiv 23 Dec 2025.
Builds on
51 of 68 references.
References · 51
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,653
- 2019Advances and Open Problems in Federated Learningcited by 3,040
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
- 2019Deep Leakage from Gradientscited by 1,017
- 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 711
- 2020Personalized Federated Learning with Moreau Envelopescited by 616
- 2018Local SGD Converges Fast and Communicates Littlecited by 601
- 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 567
- 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
- 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 517
- 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
38 more
- 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
- 2020iDLG: Improved Deep Leakage from Gradientscited by 380
- 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
- 2021See through Gradients: Image Batch Recovery via GradInversioncited by 326
- 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 324
- 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 308
- 2018The Convergence of Sparsified Gradient Methodscited by 229
- 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
- 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 179
- 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 158
- 2016Distributed Mean Estimation with Limited Communicationcited by 136
- 2020LDP-Fed: Federated Learning with Local Differential Privacycited by 132
- 2019Federated Learning for Wireless Communications: Motivation, Opportunities and Challengescited by 129
- 2018A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multiplierscited by 120
- 2024Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Modelscited by 115
- 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 107
- 2017Soft Weight-Sharing for Neural Network Compressioncited by 99
- 2019Natural Compression for Distributed Deep Learningcited by 90
- 2019Communication-efficient distributed SGD with Sketchingcited by 81
- 2020Local Differential Privacy based Federated Learning for Internet of Thingscited by 75
- 2022Gradients without Backpropagationcited by 73
- 2022Scaling Forward Gradient With Local Lossescited by 65
- 2024FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuningcited by 58
- 2020Wireless Federated Learning with Local Differential Privacycited by 52
- 2021Communication Efficiency in Federated Learning: Achievements and Challengescited by 40
- 2023pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuningcited by 38
- 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
- 2022Joint Privacy Enhancement and Quantization in Federated Learningcited by 35
- 2023Can Forward Gradient Match Backpropagation?cited by 26
- 2020FedSKETCH: Communication-Efficient and Private Federated Learning via Sketchingcited by 20
- 2023Model-free Learning with Heterogeneous Dynamical Systems: A Federated LQR Approachcited by 19
- 2022Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank?cited by 16
- 2024GI-NAS: Boosting Gradient Inversion Attacks Through Adaptive Neural Architecture Searchcited by 14
- 2021DP-SIGNSGD: When Efficiency Meets Privacy and Robustnesscited by 11
- 2021A new perspective on low-rank optimizationcited by 10
- 2022Federated Learning Using Variance Reduced Stochastic Gradient for Probabilistically Activated Agentscited by 8
- 2024Projected Forward Gradient-Guided Frank-Wolfe Algorithm via Variance Reductioncited by 5
- 2024SPD-CFL: Stepwise Parameter Dropout for Efficient Continual Federated Learningcited by 5
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.