mlsanityData to 13 Oct 2026

Riemannian Hamiltonian methods for min-max optimization on manifolds

arXiv 2204.11418 ↗First posted 25 Apr 2022math.OC cs.LG stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

In this paper, we study min-max optimization problems on Riemannian manifolds. We introduce a Riemannian Hamiltonian function, minimization of which serves as a proxy for solving the original min-max problems. Under the Riemannian Polyak--Lojasiewicz condition on the Hamiltonian function, its minimizer corresponds to the desired min-max saddle point. We also provide cases where this condition is satisfied. For geodesic-bilinear optimization in particular, solving the proxy problem leads to the correct search direction towards global optimality, which becomes challenging with the min-max formulation. To minimize the Hamiltonian function, we propose Riemannian Hamiltonian methods (RHM) and present their convergence analyses. We extend RHM to include consensus regularization and to the stochastic setting. We illustrate the efficacy of the proposed RHM in applications such as subspace robust Wasserstein distance, robust training of neural networks, and generative adversarial networks.

Key facts. Cited by 12 papers in the map (data through 13 Oct 2026). Builds on 41 of its 93 references that are in the map. First posted on arXiv 25 Apr 2022.

Builds on

41 of 93 references.

References · 41

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,379
  2. 2022Generative Adversarial Networkscited by 5,614
  3. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,619
  4. 2016Universal adversarial perturbationscited by 782
  5. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 529
  6. 2018Learning Adversarially Fair and Transferable Representationscited by 353
  7. 2019A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approachcited by 142
  8. 2019Orthogonal Convolutional Neural Networkscited by 123
  9. 2017Cycles in adversarial regularized learningcited by 122
  10. 2018The Mechanics of n-Player Differentiable Gamescited by 118
  11. 2013Manopt, a Matlab toolbox for optimization on manifoldscited by 112
  12. 2020Geoopt: Riemannian Optimization in PyTorchcited by 111
29 more
  1. 2017The Numerics of GANscited by 104
  2. 2015Reducing Overfitting in Deep Networks by Decorrelating Representationscited by 94
  3. 2016Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiationcited by 93
  4. 2017Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networkscited by 92
  5. 2019Subspace Robust Wasserstein Distancescited by 90
  6. 2016Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifoldscited by 64
  7. 2018Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?cited by 62
  8. 2016Global rates of convergence for nonconvex optimization on manifoldscited by 61
  9. 2020Projection Robust Wasserstein Distance and Riemannian Optimizationcited by 45
  10. 2019On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Gamescited by 35
  11. 2019Competitive Gradient Descentcited by 32
  12. 2020Stochastic Hamiltonian Gradient Methods for Smooth Gamescited by 32
  13. 2021On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometrycited by 32
  14. 2018Local Saddle Point Optimization: A Curvature Exploitation Approachcited by 28
  15. 2020A Riemannian Block Coordinate Descent Method for Computing the Projection Robust Wasserstein Distancecited by 27
  16. 2019Last-iterate convergence rates for min-max optimizationcited by 25
  17. 2018Adaptive regularization with cubics on manifoldscited by 17
  18. 2018McTorch, a manifold optimization library for deep learningcited by 16
  19. 2020Gradient Descent Ascent for Minimax Problems on Riemannian Manifoldscited by 16
  20. 2018Geodesic Convex Optimization: Differentiation on Manifolds, Geodesics, and Convexitycited by 13
  21. 2018Manifold Optimization Over the Set of Doubly Stochastic Matrices: A Second-Order Geometrycited by 12
  22. 2018Faster First-Order Methods for Stochastic Non-Convex Optimization on Riemannian Manifoldscited by 12
  23. 2021Riemannian conjugate gradient methods: General framework and specific algorithms with convergence analysescited by 12
  24. 2015Efficient Output Kernel Learning for Multiple Taskscited by 9
  25. 2017A dual framework for low-rank tensor completioncited by 9
  26. 2021Manifold optimization for non-linear optimal transport problemscited by 8
  27. 2021tensorflow-riemopt: A Library for Optimization on Riemannian Manifoldscited by 5
  28. 2019Orthogonal Wasserstein GANscited by 4
  29. 2014A Dai-Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditionscited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 4; 2025: 1; 2026: 1.

12 papers.

  1. 2022First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spacescited by 10
  2. 2023Decentralized Riemannian Algorithm for Nonconvex Minimax Problemscited by 7
  3. 2023Accelerated Methods for Riemannian Min-Max Optimization Ensuring Bounded Geometric Penaltiescited by 4
  4. 2024Riemannian coordinate descent algorithms on matrix manifoldscited by 4
  5. 2024A Framework for Bilevel Optimization on Riemannian Manifoldscited by 3
  6. 2025Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Methodcited by 3
  7. 2023Last-Iterate Convergence of Adaptive Riemannian Gradient Descent for Equilibrium Computationcited by 2
  8. 2023Nonnegative Low-Rank Tensor Completion via Dual Formulation with Applications to Image and Video Completioncited by 1
  9. 2023Extragradient Type Methods for Riemannian Variational Inequality Problemscited by 1
  10. 2024A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial Networkscited by 1
  11. 2024Local convergence of simultaneous min-max algorithms to differential equilibrium on Riemannian manifoldcited by 1
  12. 2026LP2DH: A Locality-Preserving Pixel-Difference Hashing Framework for Dynamic Texture Recognitioncited by 0