mlsanityData to 7 Oct 2026

FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization

arXiv 2103.03452 ↗First posted 5 Mar 2021stat.ML cs.DC cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

We develop two new algorithms, called, FedDR and asyncFedDR, for solving a fundamental nonconvex composite optimization problem in federated learning. Our algorithms rely on a novel combination between a nonconvex Douglas-Rachford splitting method, randomized block-coordinate strategies, and asynchronous implementation. They can also handle convex regularizers. Unlike recent methods in the literature, e.g., FedSplit and FedPD, our algorithms update only a subset of users at each communication round, and possibly in an asynchronous manner, making them more practical. These new algorithms can handle statistical and system heterogeneity, which are the two main challenges in federated learning, while achieving the best known communication complexity. In fact, our new algorithms match the communication complexity lower bound up to a constant factor under standard assumptions. Our numerical experiments illustrate the advantages of our methods over existing algorithms on synthetic and real datasets.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 37 of its 51 references that are in the map. First posted on arXiv 5 Mar 2021.

Builds on

37 of 51 references.

References · 37

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2018Federated Learning with Non-IID Datacited by 1,099
  7. 2019Deep Leakage from Gradientscited by 1,004
  8. 2018LEAF: A Benchmark for Federated Settingscited by 780
  9. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  10. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  11. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  12. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
25 more
  1. 2019Asynchronous Federated Optimizationcited by 264
  2. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
  3. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  4. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 152
  5. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  6. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 150
  7. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  8. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  9. 2013Communication Efficient Distributed Optimization using an Approximate Newton-type Methodcited by 131
  10. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 122
  11. 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
  12. 2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
  13. 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 106
  14. 2018SGD and Hogwild! Convergence Without the Bounded Gradients Assumptioncited by 94
  15. 2020Local SGD: Unified Theory and New Efficient Methodscited by 88
  16. 2019First Analysis of Local GD on Heterogeneous Datacited by 80
  17. 2020Federated Composite Optimizationcited by 49
  18. 2016Parallel SGD: When does averaging help?cited by 30
  19. 2015ARock: an Algorithmic Framework for Asynchronous Parallel Coordinate Updatescited by 21
  20. 2017Douglas-Rachford splitting and ADMM for nonconvex optimization: tight convergence resultscited by 21
  21. 2014Global convergence of splitting methods for nonconvex composite optimizationcited by 19
  22. 2014Douglas-Rachford splitting for nonconvex optimization with application to nonconvex feasibility problemscited by 15
  23. 2018Acceleration of Primal-Dual Methods by Preconditioning and Simple Subproblem Procedurescited by 6
  24. 2015Asynchronous Block-Iterative Primal-Dual Decomposition Methods for Monotone Inclusionscited by 4
  25. 2017A Lyapunov-type approach to convergence of the Douglas-Rachford algorithmcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 8; 2024: 9; 2025: 3; 2026: 4.

29 papers.

Sort:

  1. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  2. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 35
  3. 2022FedSysID: A Federated Approach to Sample-Efficient System Identificationcited by 15
  4. 2022Fast Composite Optimization and Statistical Recovery in Federated Learningcited by 12
  5. 2023Over-the-Air Federated Learning and Optimizationcited by 10
  6. 2023Randomized Block-Coordinate Optimistic Gradient Algorithms for Root-Finding Problemscited by 9
  7. 2023Composite federated learning with heterogeneous datacited by 7
  8. 2023DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learningcited by 6
  9. 2024Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Datacited by 5
  10. 2022Clustered Federated Learning based on Nonconvex Pairwise Fusioncited by 4
  11. 2023Federated Learning with Convex Global and Local Constraintscited by 4
  12. 2025FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Datacited by 4
17 more
  1. 2022Federated Learning Using Three-Operator ADMMcited by 3
  2. 2024Distributed Event-Based Learning via ADMMcited by 3
  3. 2023DualFL: A Duality-based Federated Learning Algorithm with Communication Acceleration in the General Convex Regimecited by 2
  4. 2023Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsificationcited by 2
  5. 2024Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approachcited by 2
  6. 2024A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needscited by 2
  7. 2024Controlling Participation in Federated Learning with Feedbackcited by 1
  8. 2026FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projectioncited by 1
  9. 2024FedMap: Iterative Magnitude-Based Pruning for Communication-Efficient Federated Learningcited by 0
  10. 2024Novel clustered federated learning based on local losscited by 0
  11. 2024Federated Frank-Wolfe Algorithmcited by 0
  12. 2024Cross-Domain Latent Factors Sharing via Implicit Matrix Factorizationcited by 0
  13. 2025Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentumcited by 0
  14. 2025An Asynchronous Decentralised Optimisation Algorithm for Nonconvex Problemscited by 0
  15. 2026Finding Structure in Continual Learningcited by 0
  16. 2026Compressed Proximal Federated Learning for Non-Convex Composite Optimization on Heterogeneous Datacited by 0
  17. 2026Task Switching Without Forgetting via Proximal Decouplingcited by 0