mlsanityData to 14 Oct 2026

FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Data

arXiv 2504.11903 ↗First posted 16 Apr 2025cs.LG cs.DC math.OCcited by 4

Not on the map. Why?

Abstract (arXiv)

Composite federated learning offers a general framework for solving machine learning problems with additional regularization terms. However, existing methods often face significant limitations: many require clients to perform computationally expensive proximal operations, and their performance is frequently vulnerable to data heterogeneity. To overcome these challenges, we propose a novel composite federated learning algorithm called FedCanon, designed to solve the optimization problems comprising a possibly non-convex loss function and a weakly convex, potentially non-smooth regularization term. By decoupling proximal mappings from local updates, FedCanon requires only a single proximal evaluation on the server per iteration, thereby reducing the overall proximal computation cost. Concurrently, it integrates control variables into local updates to mitigate the client drift arising from data heterogeneity. The entire architecture avoids the complex subproblems of primal-dual alternatives. The theoretical analysis provides the first rigorous convergence guarantees for this proximal-skipping framework in the general non-convex setting. It establishes that FedCanon achieves a sublinear convergence rate, and a linear rate under the Polyak-Lojasiewicz condition, without the restrictive bounded heterogeneity assumption. Extensive experiments demonstrate that FedCanon outperforms the state-of-the-art methods in terms of both accuracy and computational efficiency, particularly under heterogeneous data distributions.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 38 of its 56 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

38 of 56 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  6. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  7. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
  8. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  9. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  10. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  11. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  12. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 144
26 more
  1. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  2. 2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
  3. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 99
  4. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 64
  5. 2020Federated Composite Optimizationcited by 49
  6. 2022Federated Learning via Inexact ADMMcited by 44
  7. 2023Momentum Benefits Non-IID Federated Learning Simply and Provablycited by 38
  8. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 36
  9. 2018A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimizationcited by 33
  10. 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
  11. 2023Accelerating Hybrid Federated Learning Convergence under Partial Participationcited by 15
  12. 2020Variable Smoothing for Weakly Convex Composite Functionscited by 12
  13. 2022Fast Composite Optimization and Statistical Recovery in Federated Learningcited by 12
  14. 2021A Stochastic Proximal Gradient Framework for Decentralized Non-Convex Composite Optimization: Topology-Independent Sample Complexity and Communication Efficiencycited by 11
  15. 2023Compressed Decentralized Proximal Stochastic Gradient Method for Nonconvex Composite Problems with Heterogeneous Datacited by 11
  16. 2022Proximal Stochastic Recursive Momentum Methods for Nonconvex Composite Decentralized Optimizationcited by 10
  17. 2024Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailabilitycited by 9
  18. 2023Byzantine-Robust Distributed Online Learning: Taming Adversarial Participants in An Adversarial Environmentcited by 8
  19. 2022Confederated Learning: Federated Learning with Decentralized Edge Serverscited by 7
  20. 2023Composite federated learning with heterogeneous datacited by 7
  21. 2024Locally Differentially Private Online Federated Learning With Correlated Noisecited by 6
  22. 2023Decentralized Inexact Proximal Gradient Method With Network-Independent Stepsizes for Convex Composite Optimizationcited by 5
  23. 2019Communication-Censored Linearized ADMM for Decentralized Consensus Optimizationcited by 4
  24. 2022Communication Compression for Distributed Nonconvex Optimizationcited by 4
  25. 2024Generalization Error Matters in Decentralized Learning Under Byzantine Attackscited by 4
  26. 2024Communication-efficient Vertical Federated Learning via Compressed Error Feedbackcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projectioncited by 1
  2. 2026Subspace Optimization for Efficient Federated Learning under Heterogeneous Datacited by 1
  3. 2026Achieving Linear Speedup for Composite Federated Learningcited by 0
  4. 2026Compressed Proximal Federated Learning for Non-Convex Composite Optimization on Heterogeneous Datacited by 0