# Moreau Envelope Based Difference-of-weakly-Convex Reformulation and Algorithm for Bilevel Programs

arXiv: https://arxiv.org/abs/2306.16761 · HTML: https://mlsanity.com/paper/2306.16761/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 38 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 9 of 9)

- [Efficiency of minimizing compositions of convex functions and smooth maps](https://mlsanity.com/paper/1605.00125/) (2016)
- [Will Bilevel Optimizers Benefit from Loops](https://mlsanity.com/paper/2205.14224/) (2022)
- [Enhanced Bilevel Optimization via Bregman Distance](https://mlsanity.com/paper/2107.12301/) (2021)
- [Difference of convex algorithms for bilevel programs with applications in hyperparameter selection](https://mlsanity.com/paper/2102.09006/) (2021)
- [Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems](https://mlsanity.com/paper/2206.05976/) (2022)
- [Majorization-minimization procedures and convergence of SQP methods for semi-algebraic and tame programs](https://mlsanity.com/paper/1409.8147/) (2014)
- [Constraint qualifications and optimality conditions in bilevel optimization](https://mlsanity.com/paper/1910.04072/) (2019)
- [A refined convergence analysis of pDCAe with applications to simultaneous sparse recovery and outlier detection](https://mlsanity.com/paper/1804.07213/) (2018)
- [Directional necessary optimality conditions for bilevel programs](https://mlsanity.com/paper/2004.01783/) (2020)

## Built on by (most cited first, 8 of 8)

- [Constrained Bi-Level Optimization: Proximal Lagrangian Value function Approach and Hessian-free Algorithm](https://mlsanity.com/paper/2401.16164/) (2024)
- [Moreau Envelope for Nonconvex Bi-Level Optimization: A Single-loop and Hessian-free Solution Strategy](https://mlsanity.com/paper/2405.09927/) (2024)
- [First-Order Methods for Linearly Constrained Bilevel Optimization](https://mlsanity.com/paper/2406.12771/) (2024)
- [Towards Resilient Safety-driven Unlearning for Diffusion Models against Downstream Fine-tuning](https://mlsanity.com/paper/2507.16302/) (2025)
- [Select-then-differentiate: Solving Bilevel Optimization with Manifold Lower-level Solution Sets](https://mlsanity.com/paper/2605.09209/) (2026)
- [A Single-Loop Bilevel Deep Learning Method for Optimal Control of Obstacle Problems](https://mlsanity.com/paper/2601.04120/) (2026)
- [Sample Complexity Analysis for Constrained Bilevel Reinforcement Learning](https://mlsanity.com/paper/2602.00282/) (2026)
- [Efficient Hessian-Free Methods for Multi-Objective Bilevel Optimization with Nonconvex Lower Level](https://mlsanity.com/paper/2608.12704/) (2026)

## Abstract (arXiv)

Bilevel programming has emerged as a valuable tool for hyperparameter selection, a central concern in machine learning. In a recent study by Ye et al. (2023), a value function-based difference of convex algorithm was introduced to address bilevel programs. This approach proves particularly powerful when dealing with scenarios where the lower-level problem exhibits convexity in both the upper-level and lower-level variables. Examples of such scenarios include support vector machines and ℓ1 and ℓ2 regularized regression. In this paper, we significantly expand the range of applications, now requiring convexity only in the lower-level variables of the lower-level program. We present an innovative single-level difference of weakly convex reformulation based on the Moreau envelope of the lower-level problem. We further develop a sequentially convergent Inexact Proximal Difference of Weakly Convex Algorithm (iP-DwCA). To evaluate the effectiveness of the proposed iP-DwCA, we conduct numerical experiments focused on tuning hyperparameters for kernel support vector machines on simulated data.