mlsanityData to 7 Oct 2026

A second-order method landing on the Stiefel manifold via Newtonunicodex2013Schulz iteration

arXiv 2605.02838 ↗First posted 4 May 2026math.OC cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Retraction-free approaches offer attractive low-cost alternatives to Riemannian methods on the Stiefel manifold, but they are often first-order, which may limit the efficiency under high-accuracy requirements. To this end, we propose a second-order method landing on the Stiefel manifold without invoking retractions, which is proved to enjoy local quadratic (or superlinear for its inexact variant) convergence. The update consists of the sum of (i) a component tangent to the level set of the constraint-defining function that aims to reduce the objective and (ii) a component normal to the same level set that reduces the infeasibility. Specifically, we construct the normal component via Newtonunicodex2013Schulz, a fixed-point iteration for orthogonalization. Moreover, we establish a geometric connection between the Newtonunicodex2013Schulz iteration and Stiefel manifolds, in which Newtonunicodex2013Schulz moves along the normal space. For the tangent component, we formulate a modified Newton equation that incorporates Newtonunicodex2013Schulz. Numerical experiments on the orthogonal Procrustes problem, principal component analysis, and real-data independent component analysis illustrate that the proposed method performs better than the existing methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 49 references that are in the map. First posted on arXiv 4 May 2026.

Builds on

22 of 49 references.

References · 22

  1. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
  2. 2018Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?cited by 62
  3. 2019A Brief Introduction to Manifold Optimizationcited by 35
  4. 2021Fast and accurate optimization on the orthogonal manifold without retractioncited by 35
  5. 2023Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraintscited by 18
  6. 2018Parallelizable Algorithms for Optimization Problems with Orthogonality Constraintscited by 16
  7. 2023Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifoldcited by 13
  8. 2022Dissolving Constraints for Riemannian Optimizationcited by 12
  9. 2021Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Functioncited by 11
  10. 2022Optimization flows landing on the Stiefel manifoldcited by 10
  11. 2021A Penalty-free Infeasible Approach for a Class of Nonsmooth Optimization Problems over the Stiefel Manifoldcited by 8
  12. 2024Optimization without Retraction on the Random Generalized Stiefel Manifoldcited by 7
10 more
  1. 2022Computing second-order points under equality constraints: revisiting Fletcher's augmented Lagrangiancited by 6
  2. 2021Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Functioncited by 5
  3. 2020An orthogonalization-free parallelizable framework for all-electron calculations in density functional theorycited by 4
  4. 2025Distributed Retraction-Free and Communication-Efficient Optimization on the Stiefel Manifoldcited by 3
  5. 2026An Embarrassingly Simple Way to Optimize Orthogonal Matrices at Scalecited by 3
  6. 2026The Riemannian Landing Method: From projected gradient flows to SQPcited by 2
  7. 2026NS-RGS: Newton-Schulz based Riemannian gradient method for orthogonal group synchronizationcited by 2
  8. 2024Local Linear Convergence of Infeasible Optimization with Orthogonal Constraintscited by 1
  9. 2025A Simple First-Order Algorithm for Full-Rank Equality Constrained Optimizationcited by 1
  10. 2025Adaptive directional decomposition methods for nonconvex constrained optimizationcited by 1

Built on by

Not cited yet.