mlsanityData to 14 Oct 2026

Compressed Proximal Federated Learning for Non-Convex Composite Optimization on Heterogeneous Data

arXiv 2603.07654 ↗First posted 8 Mar 2026math.OC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Composite Optimization (FCO) has emerged as a promising framework for training models with structural constraints (e.g., sparsity) in distributed edge networks. However, simultaneously achieving communication efficiency and convergence robustness remains a significant challenge, particularly when dealing with non-smooth regularizers, statistical heterogeneity, and the restrictions of biased compression. To address these issues, we propose FedCEF (Federated Composite Error Feedback), a novel algorithm tailored for non-convex FCO. FedCEF introduces a decoupled proximal update scheme that separates the proximal operator from communication, enabling clients to handle non-smooth terms locally while transmitting compressed information. To mitigate the noise from aggressive quantization and the bias from non-IID data, FedCEF integrates a rigorous error feedback mechanism with control variates. Furthermore, we design a communication-efficient pre-proximal downlink strategy that allows clients to exactly reconstruct global control variables without explicit transmission. We theoretically establish that FedCEF achieves sublinear convergence to a bounded residual error under general non-convexity, which is controllable via the step size and batch size. Extensive experiments on real datasets validate FedCEF maintains competitive model accuracy even under extreme compression ratios (e.g., 1%), significantly reducing the total communication volume compared to uncompressed baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 48 references that are in the map. First posted on arXiv 8 Mar 2026.

Builds on

35 of 48 references.

References · 35

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2019Advances and Open Problems in Federated Learningcited by 3,046
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  6. 2018Federated Learning with Non-IID Datacited by 1,108
  7. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  8. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 408
  9. 2018Sparsified SGD with Memorycited by 376
  10. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  11. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 181
  12. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
23 more
  1. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 160
  2. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 144
  3. 2020On Biased Compression for Distributed Learningcited by 140
  4. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 133
  5. 2020PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimizationcited by 119
  6. 2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 101
  7. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 99
  8. 2020Federated Composite Optimizationcited by 49
  9. 2020Unified Analysis of Stochastic Gradient Methods for Composite Convex and Smooth Optimizationcited by 36
  10. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 36
  11. 2023Momentum Provably Improves Error Feedback!cited by 31
  12. 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
  13. 2020A Unified Analysis of Stochastic Gradient Methods for Nonconvex Federated Optimizationcited by 28
  14. 2021Improving Fairness for Data Valuation in Horizontal Federated Learningcited by 22
  15. 20183LC: Lightweight and Effective Traffic Compression for Distributed Machine Learningcited by 16
  16. 2021Innovation Compression for Communication-efficient Distributed Optimization with Linear Convergencecited by 12
  17. 2022Fast Composite Optimization and Statistical Recovery in Federated Learningcited by 12
  18. 2022Proximal Stochastic Recursive Momentum Methods for Nonconvex Composite Decentralized Optimizationcited by 10
  19. 2023CEDAS: A Compressed Decentralized Stochastic Gradient Method with Improved Convergencecited by 9
  20. 2021Quantized Distributed Gradient Tracking Algorithm with Linear Convergence in Directed Networkscited by 8
  21. 2023Composite federated learning with heterogeneous datacited by 7
  22. 2025Non-convex composite federated learning with heterogeneous datacited by 4
  23. 2025FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Datacited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FedSGM: A Unified Framework for Constraint Aware, Bidirectionally Compressed, Multi-Step Federated Optimization
  2. 2022Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data
  3. 2025FedSparQ: Adaptive Sparse Quantization with Error Feedback for Robust & Efficient Federated Learning
  4. 2022Communication-Efficient Federated Learning for Heterogeneous Edge Devices Based on Adaptive Gradient Quantization
  5. 2024FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models
  6. 2024FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization
  7. 2025Safe-EF: Error Feedback for Nonsmooth Constrained Optimization
  8. 2024FedFQ: Federated Learning with Fine-Grained Quantization