mlsanityData to 14 Oct 2026

A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints

arXiv 2406.10148 ↗First posted 14 Jun 2024math.OC cs.LG stat.MLcited by 7

Not on the map. Why?

Abstract (arXiv)

Interest in bilevel optimization has grown in recent years, partially due to its applications to tackle challenging machine-learning problems. Several exciting recent works have been centered around developing efficient gradient-based algorithms that can solve bilevel optimization problems with provable guarantees. However, the existing literature mainly focuses on bilevel problems either without constraints, or featuring only simple constraints that do not couple variables across the upper and lower levels, excluding a range of complex applications. Our paper studies this challenging but less explored scenario and develops a (fully) first-order algorithm, which we term BLOCC, to tackle BiLevel Optimization problems with Coupled Constraints. We establish rigorous convergence theory for the proposed algorithm and demonstrate its effectiveness on two well-known real-world applications - hyperparameter selection in support vector machine (SVM) and infrastructure planning in transportation networks using the real data from the city of Seville.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 39 of its 77 references that are in the map. First posted on arXiv 14 Jun 2024.

Builds on

39 of 77 references.

References · 39

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  3. 2014Convex Optimization: Algorithms and Complexitycited by 575
  4. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
  5. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 405
  6. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 360
  7. 2018Approximation Methods for Bilevel Programmingcited by 236
  8. 2017Forward and Reverse Gradient-Based Hyperparameter Optimizationcited by 234
  9. 2016Hyperparameter optimization with approximate gradientcited by 233
  10. 2020Bilevel Optimization: Convergence Analysis and Enhanced Designcited by 222
  11. 2018Truncated Back-propagation for Bilevel Optimizationcited by 190
  12. 2020On the Iteration Complexity of Hypergradient Computationcited by 155
27 more
  1. 2013Mini-batch Stochastic Approximation Methods for Nonconvex Stochastic Composite Optimizationcited by 143
  2. 2017A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applicationscited by 129
  3. 2021A Near-Optimal Algorithm for Stochastic Bilevel Optimization via Double-Momentumcited by 106
  4. 2022BOME! Bilevel Optimization Made Easy: A Simple First-Order Approachcited by 105
  5. 2023A Fully First-Order Method for Stochastic Bilevel Optimizationcited by 100
  6. 2023On Penalty-based Bilevel Gradient Descent Methodcited by 87
  7. 2021A Single-Timescale Method for Stochastic Bilevel Optimizationcited by 69
  8. 2021Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyondcited by 63
  9. 2016Error bounds, quadratic growth, and linear convergence of proximal methodscited by 60
  10. 2021A Value-Function-based Interior-point Method for Non-convex Bi-level Optimizationcited by 60
  11. 2021A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inversecited by 53
  12. 2023On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximationcited by 51
  13. 2024Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHFcited by 41
  14. 2022Will Bilevel Optimizers Benefit from Loopscited by 36
  15. 2021On the Convergence Theory for Hessian-Free Bilevel Algorithmscited by 35
  16. 2019Penalty Method for Inversion-Free Deep Bilevel Optimizationcited by 33
  17. 2018Linear Convergence of the Primal-Dual Gradient Method for Convex-Concave Saddle Point Problems without Strong Convexitycited by 29
  18. 2022Automatic differentiation of nonsmooth iterative algorithmscited by 25
  19. 2021Value-Function-based Sequential Minimization for Bi-level Optimizationcited by 24
  20. 2022A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequencescited by 17
  21. 2024Constrained Bi-Level Optimization: Proximal Lagrangian Value function Approach and Hessian-free Algorithmcited by 16
  22. 2022Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problemscited by 15
  23. 2023Efficient Gradient Approximation Method for Constrained Bilevel Optimizationcited by 15
  24. 2023Accelerating Inexact HyperGradient Descent for Bilevel Optimizationcited by 15
  25. 2024Overcoming Lower-Level Constraints in Bilevel Optimization: A Novel Approach with Regularized Gap Functionscited by 12
  26. 2022Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimizationcited by 10
  27. 2022A Fast and Convergent Proximal Algorithm for Regularized Nonconvex and Nonsmooth Bi-level Optimizationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025A First-order Generative Bilevel Optimization Framework for Diffusion Modelscited by 4
  2. 2025In-memory Training on Analog Devices with Limited Conductance States via Multi-tile Residual Learningcited by 3
  3. 2025A Fully First-Order Layer for Differentiable Optimizationcited by 3
  4. 2026Penalty-Based First-Order Methods for Bilevel Optimization with Minimax and Constrained Lower-Level Problemscited by 1
  5. 2025Sequential QCQP for Bilevel Optimization with Line Searchcited by 0
  6. 2025A Single-Loop First-Order Algorithm for Linearly Constrained Bilevel Optimizationcited by 0
  7. 2026Sample Complexity Analysis for Constrained Bilevel Reinforcement Learningcited by 0