# Optimization and Sampling Under Continuous Symmetry: Examples and Lie Theory

arXiv: https://arxiv.org/abs/2109.01080 · HTML: https://mlsanity.com/paper/2109.01080/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 75 references that are in the map.
- First posted on arXiv 2 Sep 2021.

## Builds on (most cited first, 9 of 9)

- [The entropic barrier: a simple and optimal universal self-concordant barrier](https://mlsanity.com/paper/1412.1587/) (2014)
- [Geodesic Convex Optimization: Differentiation on Manifolds, Geodesics, and Convexity](https://mlsanity.com/paper/1806.06373/) (2018)
- [Semidefinite descriptions of the convex hull of rotation matrices](https://mlsanity.com/paper/1403.4914/) (2014)
- [Operator Scaling via Geodesically Convex Optimization, Invariant Theory and Polynomial Identity Testing](https://mlsanity.com/paper/1804.01076/) (2018)
- [Towards a theory of non-commutative optimization: geodesic first and second order methods for moment maps and polytopes](https://mlsanity.com/paper/1910.12375/) (2019)
- [Algorithmic and optimization aspects of Brascamp-Lieb inequalities, via Operator Scaling](https://mlsanity.com/paper/1607.06711/) (2016)
- [On Geodesically Convex Formulations for the Brascamp-Lieb Constant](https://mlsanity.com/paper/1804.04051/) (2018)
- [On the computability of continuous maximum entropy distributions with applications](https://mlsanity.com/paper/2004.07403/) (2020)
- [On the Computability of Continuous Maximum Entropy Distributions: Adjoint Orbits of Lie Groups](https://mlsanity.com/paper/2011.01851/) (2020)

## Built on by (most cited first, 5 of 5)

- [Symmetry in Neural Network Parameter Spaces](https://mlsanity.com/paper/2506.13018/) (2025)
- [Symmetries, flat minima, and the conserved quantities of gradient flow](https://mlsanity.com/paper/2210.17216/) (2022)
- [Orthogonal Low-rank Adaptation in Lie Groups for Continual Learning of Large Language Models](https://mlsanity.com/paper/2509.06100/) (2025)
- [Generalized Tensor-based Parameter-Efficient Fine-Tuning via Lie Group Transformations](https://mlsanity.com/paper/2504.00851/) (2025)
- [Private Matrix Approximation and Geometry of Unitary Orbits](https://mlsanity.com/paper/2207.02794/) (2022)

## Abstract (arXiv)

In the last few years, the notion of symmetry has provided a powerful and essential lens to view several optimization or sampling problems that arise in areas such as theoretical computer science, statistics, machine learning, quantum inference, and privacy. Here, we present two examples of nonconvex problems in optimization and sampling where continuous symmetries play -- implicitly or explicitly -- a key role in the development of efficient algorithms. These examples rely on deep and hidden connections between nonconvex symmetric manifolds and convex polytopes, and are heavily generalizable. To formulate and understand these generalizations, we then present an introduction to Lie theory -- an indispensable mathematical toolkit for capturing and working with continuous symmetries. We first present the basics of Lie groups, Lie algebras, and the adjoint actions associated with them, and we also mention the classification theorem for Lie algebras. Subsequently, we present Kostant's convexity theorem and show how it allows us to reduce linear optimization problems over orbits of Lie groups to linear optimization problems over polytopes. Finally, we present the Harish-Chandra and the Harish-Chandra--Itzykson--Zuber (HCIZ) formulas, which convert partition functions (integrals) over Lie groups into sums over the corresponding (discrete) Weyl groups, enabling efficient sampling algorithms.