mlsanityData to 7 Oct 2026

A Framework for Bilevel Optimization on Riemannian Manifolds

arXiv 2402.03883 ↗First posted 6 Feb 2024math.OC cs.LG stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

Bilevel optimization has gained prominence in various applications. In this study, we introduce a framework for solving bilevel optimization problems, where the variables in both the lower and upper levels are constrained on Riemannian manifolds. We present several hypergradient estimation strategies on manifolds and analyze their estimation errors. Furthermore, we provide comprehensive convergence and complexity analyses for the proposed hypergradient descent algorithm on manifolds. We also extend our framework to encompass stochastic bilevel optimization and incorporate the use of general retraction. The efficacy of the proposed framework is demonstrated through several applications.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 47 of its 74 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

47 of 74 references.

References · 47

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  4. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  5. 2015Optimal Transport for Domain Adaptationcited by 457
  6. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  7. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  8. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  9. 2017Joint Distribution Optimal Transportation for Domain Adaptationcited by 280
  10. 2018Approximation Methods for Bilevel Programmingcited by 231
  11. 2020Bilevel Optimization: Convergence Analysis and Enhanced Designcited by 219
  12. 2016A Riemannian Network for SPD Matrix Learningcited by 162
35 more
  1. 2020On the Iteration Complexity of Hypergradient Computationcited by 154
  2. 2020A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Criticcited by 141
  3. 2020Geoopt: Riemannian Optimization in PyTorchcited by 109
  4. 2022BOME! Bilevel Optimization Made Easy: A Simple First-Order Approachcited by 103
  5. 2022A framework for bilevel optimization that enables stochastic and global variance reduction algorithmscited by 99
  6. 2023A Fully First-Order Method for Stochastic Bilevel Optimizationcited by 97
  7. 2014Stochastic Compositional Gradient Descent: Algorithms for Minimizing Compositions of Expected-Value Functionscited by 93
  8. 2016First-order Methods for Geodesically Convex Optimizationcited by 92
  9. 2020Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transformcited by 79
  10. 2021A Single-Timescale Method for Stochastic Bilevel Optimizationcited by 69
  11. 2020Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameterscited by 65
  12. 2016Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifoldscited by 64
  13. 2023On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysiscited by 39
  14. 2019Escaping from saddle points on Riemannian manifoldscited by 33
  15. 2021On the Convergence Theory for Hessian-Free Bilevel Algorithmscited by 33
  16. 2021On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometrycited by 32
  17. 2016Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methodscited by 29
  18. 2019A Continuous-time Perspective for Modeling Acceleration in Riemannian Optimizationcited by 25
  19. 2016Geometric Mean Metric Learningcited by 22
  20. 2020Gradient Descent Ascent for Minimax Problems on Riemannian Manifoldscited by 16
  21. 2018Manifold Optimization Over the Set of Doubly Stochastic Matrices: A Second-Order Geometrycited by 12
  22. 2022Riemannian Hamiltonian methods for min-max optimization on manifoldscited by 12
  23. 2021Learning with symmetric positive definite matrices via generalized Bures-Wasserstein geometrycited by 10
  24. 2022First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spacescited by 10
  25. 2022Sion's Minimax Theorem in Geodesic Metric Spaces and a Riemannian Extragradient Algorithmcited by 9
  26. 2021Manifold optimization for non-linear optimal transport problemscited by 8
  27. 2023Nystrom Method for Accurate and Scalable Implicit Differentiationcited by 8
  28. 2020Riemannian stochastic recursive momentum method for non-convex optimizationcited by 7
  29. 2020Neural Architecture Search of SPD Manifold Networkscited by 6
  30. 2022Riemannian accelerated gradient methods via extrapolationcited by 5
  31. 2023Riemannian Optimistic Algorithmscited by 5
  32. 2023Accelerated Methods for Riemannian Min-Max Optimization Ensuring Bounded Geometric Penaltiescited by 4
  33. 2022Riemannian Stochastic Gradient Method for Nested Composition Optimizationcited by 1
  34. 2023Extragradient Type Methods for Riemannian Variational Inequality Problemscited by 1
  35. 2024Theoretical smoothing frameworks for nonsmooth simple bilevel problemscited by 1

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 1.

3 papers.

  1. 2024Riemannian coordinate descent algorithms on matrix manifoldscited by 4
  2. 2025Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Methodcited by 3
  3. 2024Riemannian Bilevel Optimizationcited by 0