# Coordinate Descent Without Coordinates: Tangent Subspace Descent on Riemannian Manifolds

arXiv: https://arxiv.org/abs/1912.10627 · HTML: https://mlsanity.com/paper/1912.10627/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2019.

## Built on by (most cited first, 8 of 8)

- [Distributional Counterfactual Explanations With Optimal Transport](https://mlsanity.com/paper/2401.13112/) (2024)
- [Coordinate descent on the orthogonal group for recurrent neural network training](https://mlsanity.com/paper/2108.00051/) (2021)
- [Riemannian coordinate descent algorithms on matrix manifolds](https://mlsanity.com/paper/2406.02225/) (2024)
- [Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Method](https://mlsanity.com/paper/2505.12378/) (2025)
- [A Block Coordinate Descent Method for Nonsmooth Composite Optimization under Orthogonality Constraints](https://mlsanity.com/paper/2304.03641/) (2023)
- [Low-complexity subspace-descent over symmetric positive definite manifold](https://mlsanity.com/paper/2305.02041/) (2023)
- [Optimal low-rank stochastic gradient estimation for LLM training](https://mlsanity.com/paper/2603.20632/) (2026)
- [Convergence and Complexity Guarantee for Inexact First-order Riemannian Optimization Algorithms](https://mlsanity.com/paper/2405.03073/) (2024)

## Abstract (arXiv)

We extend coordinate descent to manifold domains, and provide convergence analyses for geodesically convex and non-convex smooth objective functions. Our key insight is to draw an analogy between coordinate blocks in Euclidean space and tangent subspaces of a manifold. Hence, our method is called tangent subspace descent (TSD). The core principle behind ensuring convergence of TSD is the appropriate choice of subspace at each iteration. To this end, we propose two novel conditions, the gap ensuring and C-randomized norm conditions on deterministic and randomized modes of subspace selection respectively, that promise convergence for smooth functions and that are satisfied in practical contexts. We propose two subspace selection rules of particular practical interest that satisfy these conditions: a deterministic one for the manifold of square orthogonal matrices, and a randomized one for the Stiefel manifold. Our proof-of-concept numerical experiments on the orthogonal Procrustes problem demonstrate TSD's efficacy.