mlsanityData to 7 Oct 2026

FLECS-CGD: A Federated Learning Second-Order Framework via Compression and Sketching with Compressed Gradient Differences

arXiv 2210.09626 ↗First posted 18 Oct 2022cs.LG math.OCcited by 2

Not on the map. Why?

Abstract (arXiv)

In the recent paper FLECS (Agafonov et al, FLECS: A Federated Learning Second-Order Framework via Compression and Sketching), the second-order framework FLECS was proposed for the Federated Learning problem. This method utilize compression of sketched Hessians to make communication costs low. However, the main bottleneck of FLECS is gradient communication without compression. In this paper, we propose the modification of FLECS with compressed gradient differences, which we call FLECS-CGD (FLECS with Compressed Gradient Differences) and make it applicable for stochastic optimization. Convergence guarantees are provided in strongly convex and nonconvex cases. Experiments show the practical benefit of proposed approach.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 28 of its 37 references that are in the map. First posted on arXiv 18 Oct 2022.

Builds on

28 of 37 references.

References · 28

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  3. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  4. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  5. 2017SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradientcited by 218
  6. 2019Distributed Learning with Compressed Gradient Differencescited by 110
  7. 2020Acceleration for Compressed Gradient Descent in Distributed and Federated Optimizationcited by 90
  8. 2019Natural Compression for Distributed Deep Learningcited by 88
  9. 2016CoCoA: A General Framework for Communication-Efficient Distributed Optimizationcited by 69
  10. 2019Stochastic Distributed Learning with Gradient Quantization and Variance Reductioncited by 64
  11. 2021FedNL: Making Newton-Type Methods Applicable to Federated Learningcited by 50
  12. 2014Communication-Efficient Distributed Dual Coordinate Ascentcited by 30
16 more
  1. 2015Fast Convergence Rates for Distributed Non-Bayesian Learningcited by 26
  2. 2013Distributed Coordinate Descent Method for Learning with Big Datacited by 23
  3. 2015Distributed Optimization with Arbitrary Local Solverscited by 23
  4. 2018Decentralize and Randomize: Faster Algorithm for Wasserstein Barycenterscited by 21
  5. 2021AI-SARAH: Adaptive and Implicit Stochastic Recursive Gradient Methodscited by 15
  6. 2017A Newton-Based Method for Nonconvex Optimization with Fast Evasion of Saddle Pointscited by 12
  7. 2018Inexact SARAH Algorithm for Stochastic Optimizationcited by 12
  8. 2022Distributed Learning With Sparsified Gradient Differencescited by 9
  9. 2019Multi-Agent Image Classification via Reinforcement Learningcited by 8
  10. 2015Communication-Efficient Distributed Optimization of Self-Concordant Empirical Losscited by 7
  11. 2021Newton Method over Networks is Fast up to the Statistical Precisioncited by 7
  12. 2021An Accelerated Second-Order Method for Distributed Stochastic Optimizationcited by 6
  13. 2021A Stochastic Newton Algorithm for Distributed Convex Optimizationcited by 6
  14. 2018Distributed Computation of Wasserstein Barycenters over Networkscited by 5
  15. 2021Hyperfast Second-Order Local Solvers for Efficient Statistically Preconditioned Distributed Optimizationcited by 5
  16. 2014Distributed Block Coordinate Descent for Minimizing Partially Separable Functionscited by 3

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 1.

2 papers.

  1. 2023Advancements in Federated Learning: Models, Methods, and Privacycited by 7
  2. 2025Simple Stepsize for Quasi-Newton Methods with Global Convergence Guaranteescited by 0