# A Framework of Constraint Preserving Update Schemes for Optimization on Stiefel Manifold

arXiv: https://arxiv.org/abs/1301.0172 · HTML: https://mlsanity.com/paper/1301.0172/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jan 2013.

## Built on by (most cited first, 13 of 13)

- [Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Function](https://mlsanity.com/paper/2112.14949/) (2021)
- [Unitary convolutions for learning on graphs and groups](https://mlsanity.com/paper/2410.05499/) (2024)
- [Gradient-descent quantum process tomography by learning Kraus operators](https://mlsanity.com/paper/2208.00812/) (2022)
- [Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs](https://mlsanity.com/paper/2205.08385/) (2022)
- [Riemannian Optimization on Relaxed Indicator Matrix Manifold](https://mlsanity.com/paper/2503.20505/) (2025)
- [Reinforcement Learning with Exogenous States and Rewards](https://mlsanity.com/paper/2303.12957/) (2023)
- [Differentiable Programming of Isometric Tensor Networks](https://mlsanity.com/paper/2110.03898/) (2021)
- [A Block Coordinate Descent Method for Nonsmooth Composite Optimization under Orthogonality Constraints](https://mlsanity.com/paper/2304.03641/) (2023)
- [LipNeXt: Scaling up Lipschitz-based Certified Robustness to Billion-parameter Models](https://mlsanity.com/paper/2601.18513/) (2026)
- [Convex optimization over a probability simplex](https://mlsanity.com/paper/2305.09046/) (2023)
- [Robust Principal Component Analysis using Density Power Divergence](https://mlsanity.com/paper/2309.13531/) (2023)
- [Retraction-Free Optimization over the Stiefel Manifold for the LoRA Fine-Tuning](https://mlsanity.com/paper/2607.25299/) (2026)
- [Retraction-Based Gradient Projection Algorithms on Manifolds](https://mlsanity.com/paper/2609.30885/) (2026)

## Abstract (arXiv)

This paper considers optimization problems on the Stiefel manifold X^mathsfTX=Ip, where X∈ ℝn × p is the variable and Ip is the p-by-p identity matrix. A framework of constraint preserving update schemes is proposed by decomposing each feasible point into the range space of X and the null space of X^mathsfT. While this general framework can unify many existing schemes, a new update scheme with low complexity cost is also discovered. Then we study a feasible Barzilai-Borwein-like method under the new update scheme. The global convergence of the method is established with an adaptive nonmonotone line search. The numerical tests on the nearest low-rank correlation matrix problem, the Kohn-Sham total energy minimization and a specific problem from statistics demonstrate the efficiency of the new method. In particular, the new method performs remarkably well for the nearest low-rank correlation matrix problem in terms of speed and solution quality and is considerably competitive with the widely used SCF iteration for the Kohn-Sham total energy minimization.