mlsanityData to 7 Oct 2026

On Penalty-based Bilevel Gradient Descent Method

arXiv 2302.05185 ↗First posted 10 Feb 2023cs.LG math.OC stat.MLcited by 87

Not on the map. Why?

Abstract (arXiv)

Bilevel optimization enjoys a wide range of applications in emerging machine learning and signal processing problems such as hyper-parameter optimization, image reconstruction, meta-learning, adversarial training, and reinforcement learning. However, bilevel optimization problems are traditionally known to be difficult to solve. Recent progress on bilevel algorithms mainly focuses on bilevel optimization problems through the lens of the implicit-gradient method, where the lower-level objective is either strongly convex or unconstrained. In this work, we tackle a challenging class of bilevel problems through the lens of the penalty method. We show that under certain conditions, the penalty reformulation recovers the (local) solutions of the original bilevel problem. Further, we propose the penalty-based bilevel gradient descent (PBGD) algorithm and establish its finite-time convergence for the constrained bilevel problem with lower-level constraints yet without lower-level strong convexity. Experiments on synthetic and real datasets showcase the efficiency of the proposed PBGD algorithm.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). Builds on 52 of its 76 references that are in the map. First posted on arXiv 10 Feb 2023.

Builds on

52 of 76 references.

References · 52

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  3. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  4. 2019Meta-Learning with Implicit Gradientscited by 523
  5. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  6. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 357
  7. 2020On the Global Convergence Rates of Softmax Policy Gradient Methodscited by 258
  8. 2017Forward and Reverse Gradient-Based Hyperparameter Optimizationcited by 232
  9. 2016Hyperparameter optimization with approximate gradientcited by 231
  10. 2018Approximation Methods for Bilevel Programmingcited by 231
  11. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 231
  12. 2018Truncated Back-propagation for Bilevel Optimizationcited by 187
38 more
  1. 2020On the Iteration Complexity of Hypergradient Computationcited by 154
  2. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  3. 2019Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methodscited by 148
  4. 2013Mini-batch Stochastic Approximation Methods for Nonconvex Stochastic Composite Optimizationcited by 142
  5. 2022Adversarially Trained Actor Critic for Offline Reinforcement Learningcited by 111
  6. 2021A Near-Optimal Algorithm for Stochastic Bilevel Optimization via Double-Momentumcited by 106
  7. 2022BOME! Bilevel Optimization Made Easy: A Simple First-Order Approachcited by 103
  8. 2022A framework for bilevel optimization that enables stochastic and global variance reduction algorithmscited by 99
  9. 2014Stochastic Compositional Gradient Descent: Algorithms for Minimizing Compositions of Expected-Value Functionscited by 93
  10. 2020A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singletoncited by 78
  11. 2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 75
  12. 2021A Single-Timescale Method for Stochastic Bilevel Optimizationcited by 69
  13. 2017A First Order Method for Solving Convex Bi-Level Optimization Problemscited by 67
  14. 2021Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyondcited by 62
  15. 2021A Value-Function-based Interior-point Method for Non-convex Bi-level Optimizationcited by 60
  16. 2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
  17. 2016Error bounds, quadratic growth, and linear convergence of proximal methodscited by 59
  18. 2016Efficiency of minimizing compositions of convex functions and smooth mapscited by 59
  19. 2021A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inversecited by 53
  20. 2023First-order penalty methods for bilevel optimizationcited by 40
  21. 2022Will Bilevel Optimizers Benefit from Loopscited by 36
  22. 2019Penalty Method for Inversion-Free Deep Bilevel Optimizationcited by 33
  23. 2021On the Convergence Theory for Hessian-Free Bilevel Algorithmscited by 33
  24. 2022Decentralized Gossip-Based Stochastic Bilevel Optimization over Communication Networkscited by 32
  25. 2022On Implicit Bias in Overparameterized Bilevel Optimizationcited by 30
  26. 2021A General Descent Aggregation Framework for Gradient-based Bi-level Optimizationcited by 28
  27. 2021Enhanced Bilevel Optimization via Bregman Distancecited by 28
  28. 2023Probabilistic Bilevel Coreset Selectioncited by 26
  29. 2022Automatic differentiation of nonsmooth iterative algorithmscited by 25
  30. 2022Non-Convex Bilevel Games with Critical Point Selection Mapscited by 25
  31. 2022Decentralized Bilevel Optimizationcited by 23
  32. 2021Tighter Analysis of Alternating Stochastic Gradient Method for Stochastic Nested Problemscited by 21
  33. 2021Bilevel methods for image reconstructioncited by 20
  34. 2018Learning to Defend by Learning to Attackcited by 17
  35. 2021Difference of convex algorithms for bilevel programs with applications in hyperparameter selectioncited by 17
  36. 2022A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequencescited by 17
  37. 2022Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problemscited by 15
  38. 2022Multi-block Min-max Bilevel Optimization with Applications in Multi-task Deep AUC Maximizationcited by 11

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 29; 2025: 29; 2026: 18.

87 papers.

Sort:

By area:

  1. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 211
  2. 2024SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selectioncited by 64
  3. 2023On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximationcited by 50
  4. 2024Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHFcited by 41
  5. 2023An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learningcited by 39
  6. 2023Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditionscited by 31
  7. 2025Beyond Two-Stage Training: Cooperative SFT and RL for LLM Reasoningcited by 28
  8. 2024Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Modelscited by 22
  9. 2024Functional Bilevel Optimization for Machine Learningcited by 20
  10. 2023Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oraclescited by 16
  11. 2024Constrained Bi-Level Optimization: Proximal Lagrangian Value function Approach and Hessian-free Algorithmcited by 16
  12. 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
38 more
  1. 2024Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysiscited by 15
  2. 2024Moreau Envelope for Nonconvex Bi-Level Optimization: A Single-loop and Hessian-free Solution Strategycited by 15
  3. 2024Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Modelscited by 15
  4. 2023On Momentum-Based Gradient Methods for Bilevel Optimization with Nonconvex Lower-Levelcited by 14
  5. 2023SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learningcited by 14
  6. 2023Projection-Free Methods for Stochastic Simple Bilevel Optimization with Convex Lower-level Problemcited by 13
  7. 2024On the Complexity of First-Order Methods in Stochastic Bilevel Optimizationcited by 11
  8. 2024Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimizationcited by 11
  9. 2024Bilevel reinforcement learning via the development of hyper-gradient without lower-level convexitycited by 10
  10. 2024Optimal Hessian/Jacobian-Free Nonconvex-PL Bilevel Optimizationcited by 10
  11. 2025On The Sample Complexity Bounds In Bilevel Reinforcement Learningcited by 10
  12. 2024An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothnesscited by 9
  13. 2024FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learningcited by 9
  14. 2024BiLO: Bilevel Local Operator Learning for PDE Inverse Problemscited by 8
  15. 2025Direct Prediction Set Minimization via Bilevel Conformal Classifier Trainingcited by 8
  16. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instancecited by 8
  17. 2024A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraintscited by 7
  18. 2024A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothnesscited by 7
  19. 2025LDC-MTL: Balancing Multi-Task Learning through Scalable Loss Discrepancy Controlcited by 7
  20. 2023A Generalized Alternating Method for Bilevel Learning under the Polyak-Lojasiewicz Conditioncited by 5
  21. 2024Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimizationcited by 5
  22. 2024Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysiscited by 5
  23. 2025Bilevel ZOFO: Efficient LLM Fine-Tuning and Meta-Trainingcited by 5
  24. 2025Riemannian Optimization on Relaxed Indicator Matrix Manifoldcited by 5
  25. 2025Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functionscited by 4
  26. 2025A First-order Generative Bilevel Optimization Framework for Diffusion Modelscited by 4
  27. 2023Effective Bilevel Optimization via Minimax Reformulationcited by 3
  28. 2024On The Global Convergence Of Online RLHF With Neural Parametrizationcited by 3
  29. 2024Unlocking TriLevel Learning with Level-Wise Zeroth Order Constraints: Distributed Algorithms and Provable Non-Asymptotic Convergencecited by 3
  30. 2025Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimizationcited by 3
  31. 2025Stochastic Bilevel Optimization with Heavy-Tailed Noisecited by 3
  32. 2025Solving bilevel optimization via sequential minimax optimizationcited by 3
  33. 2025Lower Complexity Bounds for Nonconvex-Strongly-Convex Bilevel Optimization with First-Order Oraclescited by 3
  34. 2025A Fully First-Order Layer for Differentiable Optimizationcited by 3
  35. 2023Adaptive Mirror Descent Bilevel Optimizationcited by 2
  36. 2023Achieving O(ε-1.5) Complexity in Hessian/Jacobian-free Stochastic Bilevel Optimizationcited by 2
  37. 2024Differentially Private Bilevel Optimizationcited by 2
  38. 2025qNBO: quasi-Newton Meets Bilevel Optimizationcited by 2

50 most cited of 87 All 87 (JSON)