# Local Linear Convergence of Infeasible Optimization with Orthogonal Constraints

arXiv: https://arxiv.org/abs/2412.05689 · HTML: https://mlsanity.com/paper/2412.05689/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 36 references that are in the map.
- First posted on arXiv 7 Dec 2024.

## Builds on (most cited first, 6 of 6)

- [Reducing Overfitting in Deep Networks by Decorrelating Representations](https://mlsanity.com/paper/1511.06068/) (2015)
- [Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraints](https://mlsanity.com/paper/2303.16510/) (2023)
- [Towards Riemannian Accelerated Gradient Methods](https://mlsanity.com/paper/1806.02812/) (2018)
- [Retraction-Free Decentralized Non-convex Optimization with Orthogonal Constraints](https://mlsanity.com/paper/2405.11590/) (2024)
- [Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Function](https://mlsanity.com/paper/2110.08986/) (2021)
- [A First-Order Numerical Algorithm without Matrix Operations](https://mlsanity.com/paper/2203.05027/) (2022)

## Built on by (most cited first, 1 of 1)

- [A second-order method landing on the Stiefel manifold via Newtonunicodex2013Schulz iteration](https://mlsanity.com/paper/2605.02838/) (2026)

## Abstract (arXiv)

Many classical and modern machine learning algorithms require solving optimization tasks under orthogonality constraints. Solving these tasks with feasible methods requires a gradient descent update followed by a retraction operation on the Stiefel manifold, which can be computationally expensive. Recently, an infeasible retraction-free approach, termed the landing algorithm, was proposed as an efficient alternative. Motivated by the common occurrence of orthogonality constraints in tasks such as principle component analysis and training of deep neural networks, this paper studies the landing algorithm and establishes a novel linear convergence rate for smooth non-convex functions using only a local Riemannian PL condition. Numerical experiments demonstrate that the landing algorithm performs on par with the state-of-the-art retraction-based methods with substantially reduced computational overhead.