mlsanityData to 7 Oct 2026

Riemannian coordinate descent algorithms on matrix manifolds

arXiv 2406.02225 ↗First posted 4 Jun 2024math.OC cs.LG stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

Many machine learning applications are naturally formulated as optimization problems on Riemannian manifolds. The main idea behind Riemannian optimization is to maintain the feasibility of the variables while moving along a descent direction on the manifold. This results in updating all the variables at every iteration. In this work, we provide a general framework for developing computationally efficient coordinate descent (CD) algorithms on matrix manifolds that allows updating only a few variables at every iteration while adhering to the manifold constraint. In particular, we propose CD algorithms for various manifolds such as Stiefel, Grassmann, (generalized) hyperbolic, symplectic, and symmetric positive (semi)definite. While the cost per iteration of the proposed CD algorithms is low, we further develop a more efficient variant via a first-order approximation of the objective function. We analyze their convergence and complexity, and empirically illustrate their efficacy in several applications.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 41 of its 75 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

41 of 75 references.

References · 41

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 674
  3. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  4. 2018Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometrycited by 261
  5. 2015Coordinate Descent Algorithmscited by 218
  6. 2019Orthogonal Convolutional Neural Networkscited by 122
  7. 2013Manopt, a Matlab toolbox for optimization on manifoldscited by 112
  8. 2019Trivializations for Gradient-Based Optimization on Manifoldscited by 78
  9. 2016Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifoldscited by 64
  10. 2018Averaging Stochastic Gradient Descent on Riemannian Manifoldscited by 39
  11. 2021Fast and accurate optimization on the orthogonal manifold without retractioncited by 35
  12. 2021On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometrycited by 32
29 more
  1. 2020A Riemannian Block Coordinate Descent Method for Computing the Projection Robust Wasserstein Distancecited by 27
  2. 2018Learning Multilingual Word Embeddings in Latent Metric Space: A Geometric Approachcited by 22
  3. 2023Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraintscited by 18
  4. 2018Parallelizable Algorithms for Optimization Problems with Orthogonality Constraintscited by 16
  5. 2021Differential Privacy Over Riemannian Manifoldscited by 16
  6. 2023Block Coordinate Descent on Smooth Manifolds: Convergence Theory and Twenty-One Examplescited by 15
  7. 2020Geometry-aware Domain Adaptation for Unsupervised Alignment of Word Embeddingscited by 14
  8. 2022Federated Learning on Riemannian Manifoldscited by 13
  9. 2018Manifold Optimization Over the Set of Doubly Stochastic Matrices: A Second-Order Geometrycited by 12
  10. 2022Dissolving Constraints for Riemannian Optimizationcited by 12
  11. 2022Riemannian Hamiltonian methods for min-max optimization on manifoldscited by 12
  12. 2022Differentially private Riemannian optimizationcited by 12
  13. 2017A dual framework for low-rank tensor completioncited by 9
  14. 2022Sion's Minimax Theorem in Geodesic Metric Spaces and a Riemannian Extragradient Algorithmcited by 9
  15. 2019Coordinate Descent Without Coordinates: Tangent Subspace Descent on Riemannian Manifoldscited by 8
  16. 2020Riemannian Optimization on the Symplectic Stiefel Manifoldcited by 8
  17. 2021Manifold optimization for non-linear optimal transport problemscited by 8
  18. 2022Riemannian block SPD coupling manifold and its application to optimal transportcited by 7
  19. 2019Accelerated Optimization With Orthogonality Constraintscited by 6
  20. 2021Coordinate descent on the orthogonal group for recurrent neural network trainingcited by 5
  21. 2021Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Functioncited by 5
  22. 2024Federated Learning on Riemannian Manifolds with Differential Privacycited by 5
  23. 2019Low-rank approximations of hyperbolic embeddingscited by 3
  24. 2022Optimization on the symplectic Stiefel manifold: SR decomposition-based retraction and applicationscited by 3
  25. 2024A Framework for Bilevel Optimization on Riemannian Manifoldscited by 3
  26. 2022Givens Coordinate Descent Methods for Rotation Matrix Learning in Trainable Embedding Indexescited by 2
  27. 2023A Block Coordinate Descent Method for Nonsmooth Composite Optimization under Orthogonality Constraintscited by 2
  28. 2023Low-complexity subspace-descent over symmetric positive definite manifoldcited by 2
  29. 2021Geometry of the symplectic Stiefel manifold endowed with the Euclidean metriccited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groupscited by 23
  2. 2025Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Methodcited by 3
  3. 2026An Embarrassingly Simple Way to Optimize Orthogonal Matrices at Scalecited by 3
  4. 2025Differentially Private Geodesic Regressioncited by 0