mlsanityData to 14 Oct 2026

Riemannian Bilevel Optimization

arXiv 2405.15816 ↗First posted 22 May 2024math.OC cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We develop new algorithms for Riemannian bilevel optimization. We focus in particular on batch and stochastic gradient-based methods, with the explicit goal of avoiding second-order information such as Riemannian hyper-gradients. We propose and analyze RF2SA, a method that leverages first-order gradient information to navigate the complex geometry of Riemannian manifolds efficiently. Notably, RF2SA is a single-loop algorithm, and thus easier to implement and use. Under various setups, including stochastic optimization, we provide explicit convergence rates for reaching ε-stationary points. We also address the challenge of optimizing over Riemannian manifolds with constraints by adjusting the multiplier in the Lagrangian, ensuring convergence to the desired solution without requiring access to second-order derivatives.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 50 references that are in the map. First posted on arXiv 22 May 2024.

Builds on

29 of 50 references.

References · 29

  1. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  2. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  3. 2019Meta-Learning with Implicit Gradientscited by 525
  4. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 405
  5. 2018Approximation Methods for Bilevel Programmingcited by 236
  6. 2016Hyperparameter optimization with approximate gradientcited by 233
  7. 2018Riemannian Adaptive Optimization Methodscited by 194
  8. 2020Fast Global Convergence of Natural Policy Gradient Methods with Entropy Regularizationcited by 176
  9. 2020A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Criticcited by 144
  10. 2021Hyperbolic Deep Neural Networks: A Surveycited by 139
  11. 2023A Fully First-Order Method for Stochastic Bilevel Optimizationcited by 100
  12. 2016First-order Methods for Geodesically Convex Optimizationcited by 93
17 more
  1. 2021iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradientscited by 51
  2. 2021Stability and Generalization of Bilevel Programming in Hyperparameter Optimizationcited by 31
  3. 2020A mean-field analysis of two-player zero-sum gamescited by 28
  4. 2019Bayesian Optimization Meets Riemannian Manifolds in Robot Learningcited by 23
  5. 2021ZARTS: On Zero-order Optimization for Neural Architecture Searchcited by 18
  6. 2020High-Dimensional Bayesian Optimization via Nested Riemannian Manifoldscited by 17
  7. 2024Constrained Bi-Level Optimization: Proximal Lagrangian Value function Approach and Hessian-free Algorithmcited by 16
  8. 2023Efficient Gradient Approximation Method for Constrained Bilevel Optimizationcited by 15
  9. 2020Fast Global Convergence for Low-rank Matrix Recovery via Riemannian Gradient Descent with Random Initializationcited by 9
  10. 2022Sion's Minimax Theorem in Geodesic Metric Spaces and a Riemannian Extragradient Algorithmcited by 9
  11. 2022On a Computationally Ill-Behaved Bilevel Problem with a Continuous and Nonconvex Lower Levelcited by 5
  12. 2022Riemannian Natural Gradient Methodscited by 5
  13. 2021Riemannian Perspective on Matrix Factorizationcited by 3
  14. 2024A Framework for Bilevel Optimization on Riemannian Manifoldscited by 3
  15. 2015Robust PCA: Optimization of the Robust Reconstruction Error over the Stiefel Manifoldcited by 2
  16. 2023Last-Iterate Convergence of Adaptive Riemannian Gradient Descent for Equilibrium Computationcited by 2
  17. 2024Riemannian Bilevel Optimizationcited by 2

Built on by

Not cited yet.