# Variational Optimization on Lie Groups, with Examples of Leading (Generalized) Eigenvalue Problems

arXiv: https://arxiv.org/abs/2001.10006 · HTML: https://mlsanity.com/paper/2001.10006/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2020.

## Built on by (most cited first, 12 of 12)

- [Trivialized Momentum Facilitates Diffusion Generative Modeling on Lie Groups](https://mlsanity.com/paper/2405.16381/) (2024)
- [Alternating Mirror Descent for Constrained Min-Max Games](https://mlsanity.com/paper/2206.04160/) (2022)
- [Momentum Stiefel Optimizer, with Applications to Suitably-Orthogonal Attention, and Optimal Transport](https://mlsanity.com/paper/2205.14173/) (2022)
- [Kinetic Langevin Diffusion for Crystalline Materials Generation](https://mlsanity.com/paper/2507.03602/) (2025)
- [Quantum State Generation with Structure-Preserving Diffusion Model](https://mlsanity.com/paper/2404.06336/) (2024)
- [Convergence of Kinetic Langevin Monte Carlo on Lie groups](https://mlsanity.com/paper/2403.12012/) (2024)
- [Projected Neural Differential Equations for Learning Constrained Dynamics](https://mlsanity.com/paper/2410.23667/) (2024)
- [Inverting Data Transformations via Diffusion Sampling](https://mlsanity.com/paper/2602.08267/) (2026)
- [Fast symplectic integrator for Nesterov-type acceleration method](https://mlsanity.com/paper/2106.07620/) (2021)
- [Extragradient Type Methods for Riemannian Variational Inequality Problems](https://mlsanity.com/paper/2309.14155/) (2023)
- [Quantitative Convergences of Lie Group Momentum Optimizers](https://mlsanity.com/paper/2405.20390/) (2024)
- [Momentum-based gradient descent methods for Lie groups](https://mlsanity.com/paper/2404.09363/) (2024)

## Abstract (arXiv)

The article considers smooth optimization of functions on Lie groups. By generalizing NAG variational principle in vector space (Wibisono et al., 2016) to Lie groups, continuous Lie-NAG dynamics which are guaranteed to converge to local optimum are obtained. They correspond to momentum versions of gradient flow on Lie groups. A particular case of mathsfSO(n) is then studied in details, with objective functions corresponding to leading Generalized EigenValue problems: the Lie-NAG dynamics are first made explicit in coordinates, and then discretized in structure preserving fashions, resulting in optimization algorithms with faithful energy behavior (due to conformal symplecticity) and exactly remaining on the Lie group. Stochastic gradient versions are also investigated. Numerical experiments on both synthetic data and practical problem (LDA for MNIST) demonstrate the effectiveness of the proposed methods as optimization algorithms (not as a classification method).