# Retraction-Free Decentralized Non-convex Optimization with Orthogonal Constraints

arXiv: https://arxiv.org/abs/2405.11590 · HTML: https://mlsanity.com/paper/2405.11590/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2024.

## Built on by (most cited first, 6 of 6)

- [PoLAR: Polar-Decomposed Low-Rank Adapter Representation](https://mlsanity.com/paper/2506.03133/) (2025)
- [Decentralized Online Riemannian Optimization Beyond Hadamard Manifolds](https://mlsanity.com/paper/2509.07779/) (2025)
- [Online Optimization on Hadamard Manifolds: Curvature Independent Regret Bounds on Horospherically Convex Objectives](https://mlsanity.com/paper/2509.11236/) (2025)
- [Finite-Time Analysis of Stochastic Nonconvex Nonsmooth Optimization on the Riemannian Manifolds](https://mlsanity.com/paper/2510.21468/) (2025)
- [Local Linear Convergence of Infeasible Optimization with Orthogonal Constraints](https://mlsanity.com/paper/2412.05689/) (2024)
- [A Double Tracking Method for Optimization with Decentralized Generalized Orthogonality Constraints](https://mlsanity.com/paper/2409.04998/) (2024)

## Abstract (arXiv)

In this paper, we investigate decentralized non-convex optimization with orthogonal constraints. Conventional algorithms for this setting require either manifold retractions or other types of projection to ensure feasibility, both of which involve costly linear algebra operations (e.g., SVD or matrix inversion). On the other hand, infeasible methods are able to provide similar performance with higher computational efficiency. Inspired by this, we propose the first decentralized version of the retraction-free landing algorithm, called Decentralized Retraction-Free Gradient Tracking (DRFGT). We theoretically prove that DRFGT enjoys the ergodic convergence rate of O(1/K), matching the convergence rate of centralized, retraction-based methods. We further establish that under a local Riemannian PL condition, DRFGT achieves a much faster linear convergence rate. Numerical experiments demonstrate that DRFGT performs on par with the state-of-the-art retraction-based methods with substantially reduced computational overhead.