# A space-decoupling framework for optimization on bounded-rank matrices with orthogonally invariant constraints

arXiv: https://arxiv.org/abs/2501.13830 · HTML: https://mlsanity.com/paper/2501.13830/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 62 references that are in the map.
- First posted on arXiv 23 Jan 2025.

## Builds on (most cited first, 19 of 19)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Manopt, a Matlab toolbox for optimization on manifolds](https://mlsanity.com/paper/1308.5200/) (2013)
- [The non-convex Burer-Monteiro approach works on smooth semidefinite programs](https://mlsanity.com/paper/1606.04970/) (2016)
- [Finding Low-Rank Solutions via Non-Convex Matrix Factorization, Efficiently and Provably](https://mlsanity.com/paper/1606.03168/) (2016)
- [The effect of smooth parametrizations on nonconvex optimization landscapes](https://mlsanity.com/paper/2207.03512/) (2022)
- [A Riemannian low-rank method for optimization over semidefinite matrices with block-diagonal constraints](https://mlsanity.com/paper/1506.00575/) (2015)
- [Learning Markov models via low-rank optimization](https://mlsanity.com/paper/1907.00113/) (2019)
- [Convergence results for projected line-search methods on varieties of low-rank matrices via Lojasiewicz inequality](https://mlsanity.com/paper/1402.5284/) (2014)
- [From the simplex to the sphere: Faster constrained optimization using the Hadamard parametrization](https://mlsanity.com/paper/2112.05273/) (2021)
- [Finding stationary points on bounded-rank matrices: A geometric hurdle and a smooth remedy](https://mlsanity.com/paper/2107.03877/) (2021)
- [Benign landscapes of low-dimensional relaxations for orthogonal synchronization on general graphs](https://mlsanity.com/paper/2307.02941/) (2023)
- [A Riemannian rank-adaptive method for low-rank matrix completion](https://mlsanity.com/paper/2103.14768/) (2021)
- [Normal Cones Intersection Rule and Optimality Analysis for Low-Rank Matrix Optimization with Affine Manifolds](https://mlsanity.com/paper/2202.08464/) (2022)
- [Projected gradient descent accumulates at Bouligand stationary points](https://mlsanity.com/paper/2403.02530/) (2024)
- [An apocalypse-free first-order low-rank optimization algorithm with at most one rank reduction attempt per iteration](https://mlsanity.com/paper/2208.12051/) (2022)
- [Computing Bouligand stationary points efficiently in low-rank optimization](https://mlsanity.com/paper/2409.12298/) (2024)
- [Regularity Properties of Non-Negative Sparsity Sets](https://mlsanity.com/paper/1607.05924/) (2016)
- [Optimization over bounded-rank matrices through a desingularization enables joint global and local guarantees](https://mlsanity.com/paper/2406.14211/) (2024)

## Built on by (most cited first, 2 of 2)

- [Efficient Generative Modeling with Unitary Matrix Product States Using Riemannian Optimization](https://mlsanity.com/paper/2603.12026/) (2026)
- [Optimization over the intersection of manifolds](https://mlsanity.com/paper/2605.22736/) (2026)

## Abstract (arXiv)

Imposing additional constraints on low-rank optimization has garnered growing interest. However, the geometry of coupled constraints hampers the well-developed low-rank structure and makes the problem intricate. To this end, we propose a space-decoupling framework for optimization on bounded-rank matrices with orthogonally invariant constraints. The "space-decoupling" is reflected in several ways. We show that the tangent cone of coupled constraints is the intersection of tangent cones of each constraint. Moreover, we decouple the intertwined bounded-rank and orthogonally invariant constraints into two spaces, leading to optimization on a smooth manifold. Implementing Riemannian algorithms on this manifold is painless as long as the geometry of additional constraints is known. In addition, we unveil the equivalence between the reformulated problem and the original problem. Numerical experiments on real-world applications -- spherical data fitting, graph similarity measuring, low-rank SDP, model reduction of Markov processes, reinforcement learning, and deep learning -- validate the superiority of the proposed framework.