# Select-then-differentiate: Solving Bilevel Optimization with Manifold Lower-level Solution Sets

arXiv: https://arxiv.org/abs/2605.09209 · HTML: https://mlsanity.com/paper/2605.09209/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 78 references that are in the map.
- First posted on arXiv 9 May 2026.

## Builds on (most cited first, 10 of 10)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [Spurious Valleys in Two-layer Neural Network Optimization Landscapes](https://mlsanity.com/paper/1802.06384/) (2018)
- [Moreau Envelope Based Difference-of-weakly-Convex Reformulation and Algorithm for Bilevel Programs](https://mlsanity.com/paper/2306.16761/) (2023)
- [Set Smoothness Unlocks Clarke Hyper-stationarity in Bilevel Optimization](https://mlsanity.com/paper/2506.04587/) (2025)
- [Superquantile-Gibbs Relaxation for Minima-selection in Bilevel Optimization](https://mlsanity.com/paper/2505.05991/) (2025)
- [A Correspondence-Driven Approach for Bilevel Decision-making with Nonconvex Lower-Level Problems](https://mlsanity.com/paper/2509.01148/) (2025)
- [Bilevel gradient methods and the Morse parametric qualification condition](https://mlsanity.com/paper/2502.09074/) (2025)
- [Stable Preference Optimization: A Bilevel Approach to Catastrophic Preference Shift](https://mlsanity.com/paper/2507.07723/) (2025)

## Built on by (most cited first, 2 of 2)

- [Cross-View Correspondence Is a Measurement Intervention: Two-Sided Validation for Agent Evaluation and Credit Assignment](https://mlsanity.com/paper/2608.17713/) (2026)
- [Gauss-Newton Accuracy and Indefinite Hessians: Uniform Coexistence in Low-Cost Sets](https://mlsanity.com/paper/2610.09675/) (2026)

## Abstract (arXiv)

We study optimistic bilevel optimization when the lower-level problem has a non-isolated manifold of minimizers. In this setting, the hyper-objective may be non-differentiable because the upper-level criterion must choose among multiple lower-level solutions. Under a local Polyak--Lojasiewicz (PL) condition, we show that differentiability does not require the lower-level solution set to be a singleton: uniqueness of the optimistic selection is sufficient. This yields an explicit pseudoinverse-based hyper-gradient formula extending the classical singleton-minimizer result. We further characterize the regularity of the hyper-objective: non-degeneracy of the selected minimizer along the solution manifold yields local smoothness, while failure of uniqueness can create many non-differentiable points and failure of non-degeneracy can destroy all positive H"older regularity of the hyper-gradient. Motivated by this theory, we propose HG-MS, a select-then-differentiate method combining explicit optimistic selection with efficient pseudoinverse-based hyper-gradient computation. Despite the nonconvex nature of optimistic selection over the lower-level solution manifold, we show that HG-MS converges to a stationary point of the optimistic objective with complexity governed by the intrinsic dimension of the solution manifold rather than its ambient dimension. Empirically, we test a practical variant of HG-MS for matched-budget LLM source reweighting. This variant preserves the select-then-differentiate principle and obtains the best GSM8K/MATH scores across the tested backbones, along with competitive or best MT-Bench instruction-following results.