# Semiparametric Efficient Bilevel Gradient Estimation

arXiv: https://arxiv.org/abs/2605.21341 · HTML: https://mlsanity.com/paper/2605.21341/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 62 references that are in the map.
- First posted on arXiv 20 May 2026.

## Builds on (most cited first, 30 of 30)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [On Exact Computation with an Infinitely Wide Neural Net](https://mlsanity.com/paper/1904.11955/) (2019)
- [Train faster, generalize better: Stability of stochastic gradient descent](https://mlsanity.com/paper/1509.01240/) (2015)
- [Meta-Learning with Implicit Gradients](https://mlsanity.com/paper/1909.04630/) (2019)
- [Bilevel Programming for Hyperparameter Optimization and Meta-Learning](https://mlsanity.com/paper/1806.04910/) (2018)
- [Gradient-based Hyperparameter Optimization through Reversible Learning](https://mlsanity.com/paper/1502.03492/) (2015)
- [Optimizing Millions of Hyperparameters by Implicit Differentiation](https://mlsanity.com/paper/1911.02590/) (2019)
- [Approximation Methods for Bilevel Programming](https://mlsanity.com/paper/1802.02246/) (2018)
- [Hyperparameter optimization with approximate gradient](https://mlsanity.com/paper/1602.02355/) (2016)
- [Bilevel Optimization: Convergence Analysis and Enhanced Design](https://mlsanity.com/paper/2010.07962/) (2020)
- [Orthogonal Statistical Learning](https://mlsanity.com/paper/1901.09036/) (2019)
- [A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic](https://mlsanity.com/paper/2007.05170/) (2020)
- [Semiparametric doubly robust targeted double machine learning: a review](https://mlsanity.com/paper/2203.06469/) (2022)
- [Kernel Instrumental Variable Regression](https://mlsanity.com/paper/1906.00232/) (2019)
- [Deep Generalized Method of Moments for Instrumental Variable Analysis](https://mlsanity.com/paper/1905.12495/) (2019)
- [RieszNet and ForestRiesz: Automatic Debiased Machine Learning with Neural Nets and Random Forests](https://mlsanity.com/paper/2110.03031/) (2021)
- [What game are we playing? End-to-end learning in normal and extensive form games](https://mlsanity.com/paper/1805.02777/) (2018)
- [End-to-End Learning and Intervention in Games](https://mlsanity.com/paper/2010.13834/) (2020)
- [Functional Bilevel Optimization for Machine Learning](https://mlsanity.com/paper/2403.20233/) (2024)
- [One-Step Estimation of Differentiable Hilbert-Valued Parameters](https://mlsanity.com/paper/2303.16711/) (2023)
- [An Omnibus Nonparametric Test of Equality in Distribution for Unknown Functions](https://mlsanity.com/paper/1510.04195/) (2015)
- [Automatic Debiased Machine Learning for Smooth Functionals of Nonparametric M-Estimands](https://mlsanity.com/paper/2501.11868/) (2025)
- [A Researcher's Guide to Empirical Risk Minimization](https://mlsanity.com/paper/2602.21501/) (2026)
- [Doubly-Robust Estimation of Counterfactual Policy Mean Embeddings](https://mlsanity.com/paper/2506.02793/) (2025)
- [Learning Theory for Kernel Bilevel Optimization](https://mlsanity.com/paper/2502.08457/) (2025)
- [Functional Natural Policy Gradients](https://mlsanity.com/paper/2603.28681/) (2026)
- [Semiparametric Efficient Test for Interpretable Distributional Treatment Effects](https://mlsanity.com/paper/2605.08034/) (2026)

## Abstract (arXiv)

Functional bilevel methods estimate a lower-level function and plug it into a hypergradient, but this plug-in gradient can retain first-order bias when the lower-level problem is learned nonparametrically. To remove this bias, we develop a semiparametric debiasing theory for population bilevel gradients based on the efficient influence function. This perspective leads to a cross-fitted orthogonal hypergradient estimator for which we establish asymptotic normality together with uniform control over the outer parameter. Under quadratic losses, the estimator reduces to a simple doubly robust score based on conditional mean nuisances. On synthetic bilevel benchmarks with known ground truth, the method tracks the oracle efficient-gradient benchmark and improves over plug-in functional hypergradients and regularized kernel bilevel baselines.