mlsanityData to 14 Oct 2026

Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Data

arXiv 2406.08465 ↗First posted 12 Jun 2024cs.LG cs.DCcited by 5

Not on the map. Why?

Abstract (arXiv)

Many machine learning tasks, such as principal component analysis and low-rank matrix completion, give rise to manifold optimization problems. Although there is a large body of work studying the design and analysis of algorithms for manifold optimization in the centralized setting, there are currently very few works addressing the federated setting. In this paper, we consider nonconvex federated learning over a compact smooth submanifold in the setting of heterogeneous client data. We propose an algorithm that leverages stochastic Riemannian gradients and a manifold projection operator to improve computational efficiency, uses local updates to improve communication efficiency, and avoids client drift. Theoretically, we show that our proposed algorithm converges sub-linearly to a neighborhood of a first-order optimal solution by using a novel analysis that jointly exploits the manifold structure and properties of the loss functions. Numerical experiments demonstrate that our algorithm has significantly smaller computational and communication overhead than existing methods.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 32 of its 48 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

32 of 48 references.

References · 32

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2019Advances and Open Problems in Federated Learningcited by 3,046
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  5. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  6. 2020Provable Meta-Learning of Linear Representationscited by 161
  7. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 154
  8. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 123
  9. 2013Manopt, a Matlab toolbox for optimization on manifoldscited by 113
  10. 2016Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiationcited by 94
  11. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  12. 2020Federated Composite Optimizationcited by 49
20 more
  1. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 36
  2. 2019A Brief Introduction to Manifold Optimizationcited by 35
  3. 2021Decentralized Riemannian Gradient Descent on the Stiefel Manifoldcited by 33
  4. 2023Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representationscited by 31
  5. 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
  6. 2022Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task modelscited by 29
  7. 2021DeEPCA: Decentralized Exact PCA with Linear Convergence Ratecited by 18
  8. 2022Federated Learning on Riemannian Manifoldscited by 13
  9. 2018Faster First-Order Methods for Stochastic Non-Convex Optimization on Riemannian Manifoldscited by 12
  10. 2022Fast Composite Optimization and Statistical Recovery in Federated Learningcited by 12
  11. 2021Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Functioncited by 11
  12. 2023Decentralized projected Riemannian gradient method for smooth optimization on compact submanifoldscited by 9
  13. 2023Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifoldcited by 7
  14. 2023Composite federated learning with heterogeneous datacited by 7
  15. 2021Distributed Principal Component Analysis with Limited Communicationcited by 6
  16. 2022Federated PCA on Grassmann Manifold for Anomaly Detection in IoT Networkscited by 5
  17. 2023Deep neural networks architectures from the perspective of manifold learningcited by 5
  18. 2024Federated Learning on Riemannian Manifolds with Differential Privacycited by 5
  19. 2020Stochastic optimization over proximally smooth setscited by 4
  20. 2019Error bound conditions and convergence of optimization methods on smooth and proximally smooth manifoldscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2024Riemannian Federated Learning via Averaging Gradient Streamscited by 4
  2. 2026FedSPDnet: Geometry-Aware Federated Deep Learning with SPDnetcited by 2
  3. 2025Federated Learning on Riemannian Manifolds: A Gradient-Free Projection-Based Approachcited by 1
  4. 2025Adaptive Aggregation with Two Gains in QFLcited by 1
  5. 2026Achieving Linear Speedup for Composite Federated Learningcited by 0