mlsanityData to 14 Oct 2026

A space-decoupling framework for optimization on bounded-rank matrices with orthogonally invariant constraints

arXiv 2501.13830 ↗First posted 23 Jan 2025math.OC cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Imposing additional constraints on low-rank optimization has garnered growing interest. However, the geometry of coupled constraints hampers the well-developed low-rank structure and makes the problem intricate. To this end, we propose a space-decoupling framework for optimization on bounded-rank matrices with orthogonally invariant constraints. The "space-decoupling" is reflected in several ways. We show that the tangent cone of coupled constraints is the intersection of tangent cones of each constraint. Moreover, we decouple the intertwined bounded-rank and orthogonally invariant constraints into two spaces, leading to optimization on a smooth manifold. Implementing Riemannian algorithms on this manifold is painless as long as the geometry of additional constraints is known. In addition, we unveil the equivalence between the reformulated problem and the original problem. Numerical experiments on real-world applications -- spherical data fitting, graph similarity measuring, low-rank SDP, model reduction of Markov processes, reinforcement learning, and deep learning -- validate the superiority of the proposed framework.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 19 of its 62 references that are in the map. First posted on arXiv 23 Jan 2025.

Builds on

19 of 62 references.

References · 19

  1. 2016OpenAI Gymcited by 1,736
  2. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 543
  3. 2013Manopt, a Matlab toolbox for optimization on manifoldscited by 113
  4. 2016The non-convex Burer-Monteiro approach works on smooth semidefinite programscited by 46
  5. 2016Finding Low-Rank Solutions via Non-Convex Matrix Factorization, Efficiently and Provablycited by 22
  6. 2022The effect of smooth parametrizations on nonconvex optimization landscapescited by 21
  7. 2015A Riemannian low-rank method for optimization over semidefinite matrices with block-diagonal constraintscited by 16
  8. 2019Learning Markov models via low-rank optimizationcited by 12
  9. 2014Convergence results for projected line-search methods on varieties of low-rank matrices via Lojasiewicz inequalitycited by 10
  10. 2021From the simplex to the sphere: Faster constrained optimization using the Hadamard parametrizationcited by 10
  11. 2021Finding stationary points on bounded-rank matrices: A geometric hurdle and a smooth remedycited by 6
  12. 2023Benign landscapes of low-dimensional relaxations for orthogonal synchronization on general graphscited by 6
7 more
  1. 2021A Riemannian rank-adaptive method for low-rank matrix completioncited by 4
  2. 2022Normal Cones Intersection Rule and Optimality Analysis for Low-Rank Matrix Optimization with Affine Manifoldscited by 3
  3. 2024Projected gradient descent accumulates at Bouligand stationary pointscited by 3
  4. 2022An apocalypse-free first-order low-rank optimization algorithm with at most one rank reduction attempt per iterationcited by 2
  5. 2024Computing Bouligand stationary points efficiently in low-rank optimizationcited by 2
  6. 2016Regularity Properties of Non-Negative Sparsity Setscited by 1
  7. 2024Optimization over bounded-rank matrices through a desingularization enables joint global and local guaranteescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Efficient Generative Modeling with Unitary Matrix Product States Using Riemannian Optimizationcited by 0
  2. 2026Optimization over the intersection of manifoldscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Rotated Manifold Optimization for Low-Rank Adaptation
  2. 2025StelLA: Subspace Learning in Low-rank Adaptation using Stiefel Manifold
  3. 2025Efficient Over-parameterized Matrix Sensing from Noisy Measurements via Alternating Preconditioned Gradient Descent
  4. 2024On the Crucial Role of Initialization for Matrix Factorization
  5. 2025Landing with the Score: Riemannian Optimization through Denoising
  6. 2025High-Dimensional Search, Low-Dimensional Solution: Decoupling Optimization from Representation
  7. 2025A Universal Banach--Bregman Framework for Stochastic Iterations: Unifying Stochastic Mirror Descent, Learning and LLM Training
  8. 2021Efficient Randomized Subspace Embeddings for Distributed Optimization under a Communication Budget