A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needs
Not on the map. Why?
Abstract (arXiv)
As a popular paradigm for juggling data privacy and collaborative training, federated learning (FL) is flourishing to distributively process the large scale of heterogeneous datasets on edged clients. Due to bandwidth limitations and security considerations, it ingeniously splits the original problem into multiple subproblems to be solved in parallel, which empowers primal dual solutions to great application values in FL. In this paper, we review the recent development of classical federated primal dual methods and point out a serious common defect of such methods in non-convex scenarios, which we say is a "dual drift" caused by dual hysteresis of those longstanding inactive clients under partial participation training. To further address this problem, we propose a novel Aligned Federated Primal Dual (A-FedPD) method, which constructs virtual dual updates to align global consensus and local dual variables for those protracted unparticipated local clients. Meanwhile, we provide a comprehensive analysis of the optimization and generalization efficiency for the A-FedPD method on smooth non-convex objectives, which confirms its high efficiency and practicality. Extensive experiments are conducted on several classical FL setups to validate the effectiveness of our proposed method.
Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 45 of its 63 references that are in the map. First posted on arXiv 27 Sep 2024.
Builds on
45 of 63 references.
References · 45
- 2015Deep Residual Learning for Image Recognitioncited by 51,950
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,653
- 2018Federated Optimization in Heterogeneous Networkscited by 2,981
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,411
- 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 903
- 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 846
- 2021Federated Learning Based on Dynamic Regularizationcited by 559
- 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 549
- 2019Agnostic Federated Learningcited by 453
- 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 144
- 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
- 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
33 more
- 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
- 2020Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descentcited by 118
- 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
- 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
- 2021FedCM: Federated Learning with Client-level Momentumcited by 77
- 2020From Local SGD to Local Fixed-Point Methods for Federated Learningcited by 74
- 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 64
- 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
- 2020Federated Composite Optimizationcited by 49
- 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
- 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
- 2022Federated Learning via Inexact ADMMcited by 44
- 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 35
- 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
- 2022RandProx: Primal-Dual Optimization Algorithms with Randomized Proximal Updatescited by 28
- 2024Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimizationcited by 28
- 2020FedADC: Accelerated Federated Learning with Drift Controlcited by 26
- 2022FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneitycited by 25
- 2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedupcited by 23
- 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!cited by 22
- 2022FedShuffle: Recipes for Better Use of Local Work in Federated Learningcited by 21
- 2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initializationcited by 21
- 2022FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregationcited by 16
- 2024Federated Learning with Bilateral Curation for Partially Class-Disjoint Datacited by 13
- 2024Domain-Inspired Sharpness-Aware Minimization Under Domain Shiftscited by 12
- 2023Enhance Local Consistency in Federated Learning: A Multi-Step Inertial Momentum Approachcited by 11
- 2022Beyond ADMM: A Unified Client-variance-reduced Adaptive Federated Learning Frameworkcited by 9
- 2023Improving Accelerated Federated Learning with Compression and Importance Samplingcited by 8
- 2021FedHybrid: A Hybrid Primal-Dual Algorithm Framework for Federated Optimizationcited by 7
- 2023Window-based Model Averaging Improves Generalization in Heterogeneous Federated Learningcited by 7
- 2023DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learningcited by 6
- 2020Federated Learning From Big Data Over Networkscited by 4
- 2023Information-Theoretic Generalization Analysis for Topology-aware Heterogeneous Federated Edge Learning over Noisy Channelscited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.