mlsanityData to 14 Oct 2026

Federated Composite Optimization

arXiv 2011.08474 ↗First posted 17 Nov 2020cs.LG cs.DC math.OCcited by 49

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a distributed learning paradigm that scales on-device learning collaboratively and privately. Standard FL algorithms such as FedAvg are primarily geared towards smooth unconstrained settings. In this paper, we study the Federated Composite Optimization (FCO) problem, in which the loss function contains a non-smooth regularizer. Such problems arise naturally in FL applications that involve sparsity, low-rank, monotonicity, or more general constraints. We first show that straightforward extensions of primal algorithms such as FedAvg are not well-suited for FCO since they suffer from the "curse of primal averaging," resulting in poor convergence. As a solution, we propose a new primal-dual algorithm, Federated Dual Averaging (FedDualAvg), which by employing a novel server dual averaging procedure circumvents the curse of primal averaging. Our theoretical analysis and empirical experiments demonstrate that FedDualAvg outperforms the other baselines.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 9; 2023: 16; 2024: 4; 2025: 6; 2026: 6.

49 papers.

Sort:

By area:

  1. 2021A Field Guide to Federated Optimizationcited by 315
  2. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
  3. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  4. 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
  5. 2022On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyondcited by 38
  6. 2021Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devicescited by 36
  7. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 36
  8. 2021Proximal and Federated Random Reshufflingcited by 33
  9. 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
  10. 2023Understanding Generalization of Federated Learning via Stability: Heterogeneity Matterscited by 27
  11. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  12. 2022A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networkscited by 20
37 more
  1. 2022SphereFed: Hyperspherical Federated Learningcited by 19
  2. 2022Rate-Distortion Theoretic Bounds on Generalization Error for Distributed Learningcited by 14
  3. 2022Fast Composite Optimization and Statistical Recovery in Federated Learningcited by 12
  4. 2021FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learningcited by 11
  5. 2022Federated Data Analytics: A Study on Linear Modelscited by 9
  6. 2023Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often!cited by 8
  7. 2023FedSampling: A Better Sampling Strategy for Federated Learningcited by 8
  8. 2023EPISODE: Episodic Gradient Clipping with Periodic Resampled Corrections for Federated Learning with Heterogeneous Datacited by 7
  9. 2023Advancements in Federated Learning: Models, Methods, and Privacycited by 7
  10. 2023Composite federated learning with heterogeneous datacited by 7
  11. 2023Federated Online and Bandit Convex Optimizationcited by 7
  12. 2024Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Datacited by 6
  13. 2024Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Datacited by 5
  14. 2023Riemannian Low-Rank Model Compression for Federated Learning with Over-the-Air Aggregationcited by 4
  15. 2023Federated Learning with Convex Global and Local Constraintscited by 4
  16. 2025FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Datacited by 4
  17. 2022A dual approach for federated learningcited by 3
  18. 2025Stacey: Promoting Stochastic Steepest Descent via Accelerated ℓp-Smooth Nonconvex Optimizationcited by 3
  19. 2021Statistical Estimation and Inference via Local SGD in Federated Learningcited by 2
  20. 2023FedDA: Faster Framework of Local Adaptive Gradient Methods via Restarted Dual Averagingcited by 2
  21. 2023DualFL: A Duality-based Federated Learning Algorithm with Communication Acceleration in the General Convex Regimecited by 2
  22. 2023Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsificationcited by 2
  23. 2024A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needscited by 2
  24. 2023Federated Gradient Matching Pursuitcited by 1
  25. 2025LoGoFair: Post-Processing for Local and Global Fairness in Federated Learningcited by 1
  26. 2026FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projectioncited by 1
  27. 2023Federated Composite Saddle Point Optimizationcited by 0
  28. 2023Heterogeneous Federated Learning Using Knowledge Codistillationcited by 0
  29. 2024Federated Frank-Wolfe Algorithmcited by 0
  30. 2025Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentumcited by 0
  31. 2025FedDuA: Doubly Adaptive Federated Learningcited by 0
  32. 2025FedFACT: A Provable Framework for Controllable Group-Fairness Calibration in Federated Learningcited by 0
  33. 2026Achieving Linear Speedup for Composite Federated Learningcited by 0
  34. 2026Compressed Proximal Federated Learning for Non-Convex Composite Optimization on Heterogeneous Datacited by 0
  35. 2026A Locally Penalized Cross-Estimate Federated Method with Guarantees for Constrained Personalized Learningcited by 0
  36. 2026Federated Rule Ensemble Method in Medical Datacited by 0
  37. 2026FedSEA: Achieving Benefit of Parallelization in Federated Online Learningcited by 0