# A second-order method landing on the Stiefel manifold via Newtonunicodex2013Schulz iteration

arXiv: https://arxiv.org/abs/2605.02838 · HTML: https://mlsanity.com/paper/2605.02838/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 4 May 2026.

## Builds on (most cited first, 22 of 22)

- [Parseval Networks: Improving Robustness to Adversarial Examples](https://mlsanity.com/paper/1704.08847/) (2017)
- [Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?](https://mlsanity.com/paper/1810.09102/) (2018)
- [A Brief Introduction to Manifold Optimization](https://mlsanity.com/paper/1906.05450/) (2019)
- [Fast and accurate optimization on the orthogonal manifold without retraction](https://mlsanity.com/paper/2102.07432/) (2021)
- [Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraints](https://mlsanity.com/paper/2303.16510/) (2023)
- [Parallelizable Algorithms for Optimization Problems with Orthogonality Constraints](https://mlsanity.com/paper/1810.03930/) (2018)
- [Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold](https://mlsanity.com/paper/2303.09261/) (2023)
- [Dissolving Constraints for Riemannian Optimization](https://mlsanity.com/paper/2203.10319/) (2022)
- [Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Function](https://mlsanity.com/paper/2112.14949/) (2021)
- [Optimization flows landing on the Stiefel manifold](https://mlsanity.com/paper/2202.09058/) (2022)
- [A Penalty-free Infeasible Approach for a Class of Nonsmooth Optimization Problems over the Stiefel Manifold](https://mlsanity.com/paper/2103.03514/) (2021)
- [Optimization without Retraction on the Random Generalized Stiefel Manifold](https://mlsanity.com/paper/2405.01702/) (2024)
- [Computing second-order points under equality constraints: revisiting Fletcher's augmented Lagrangian](https://mlsanity.com/paper/2204.01448/) (2022)
- [Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Function](https://mlsanity.com/paper/2110.08986/) (2021)
- [An orthogonalization-free parallelizable framework for all-electron calculations in density functional theory](https://mlsanity.com/paper/2007.14228/) (2020)
- [Distributed Retraction-Free and Communication-Efficient Optimization on the Stiefel Manifold](https://mlsanity.com/paper/2506.02879/) (2025)
- [An Embarrassingly Simple Way to Optimize Orthogonal Matrices at Scale](https://mlsanity.com/paper/2602.14656/) (2026)
- [The Riemannian Landing Method: From projected gradient flows to SQP](https://mlsanity.com/paper/2603.24309/) (2026)
- [NS-RGS: Newton-Schulz based Riemannian gradient method for orthogonal group synchronization](https://mlsanity.com/paper/2604.07372/) (2026)
- [Local Linear Convergence of Infeasible Optimization with Orthogonal Constraints](https://mlsanity.com/paper/2412.05689/) (2024)
- [A Simple First-Order Algorithm for Full-Rank Equality Constrained Optimization](https://mlsanity.com/paper/2510.16390/) (2025)
- [Adaptive directional decomposition methods for nonconvex constrained optimization](https://mlsanity.com/paper/2511.03210/) (2025)

## Abstract (arXiv)

Retraction-free approaches offer attractive low-cost alternatives to Riemannian methods on the Stiefel manifold, but they are often first-order, which may limit the efficiency under high-accuracy requirements. To this end, we propose a second-order method landing on the Stiefel manifold without invoking retractions, which is proved to enjoy local quadratic (or superlinear for its inexact variant) convergence. The update consists of the sum of (i) a component tangent to the level set of the constraint-defining function that aims to reduce the objective and (ii) a component normal to the same level set that reduces the infeasibility. Specifically, we construct the normal component via Newtonunicodex2013Schulz, a fixed-point iteration for orthogonalization. Moreover, we establish a geometric connection between the Newtonunicodex2013Schulz iteration and Stiefel manifolds, in which Newtonunicodex2013Schulz moves along the normal space. For the tangent component, we formulate a modified Newton equation that incorporates Newtonunicodex2013Schulz. Numerical experiments on the orthogonal Procrustes problem, principal component analysis, and real-data independent component analysis illustrate that the proposed method performs better than the existing methods.