# Optimally taming biases in black-box models for efficient semiparametric estimation

arXiv: https://arxiv.org/abs/2606.06368 · HTML: https://mlsanity.com/paper/2606.06368/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 62 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Builds on (most cited first, 29 of 29)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Confidence Intervals and Hypothesis Testing for High-Dimensional Regression](https://mlsanity.com/paper/1306.3171/) (2013)
- [Double/Debiased/Neyman Machine Learning of Treatment Effects](https://mlsanity.com/paper/1701.08687/) (2017)
- [Approximate Residual Balancing: De-Biased Inference of Average Treatment Effects in High Dimensions](https://mlsanity.com/paper/1604.07125/) (2016)
- [Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations](https://mlsanity.com/paper/1309.4686/) (2013)
- [De-biasing the Lasso: Optimal Sample Size for Gaussian Designs](https://mlsanity.com/paper/1508.02757/) (2015)
- [Adversarial Estimation of Riesz Representers](https://mlsanity.com/paper/2101.00009/) (2020)
- [Covariate Balancing Propensity Score by Tailored Loss Functions](https://mlsanity.com/paper/1601.05890/) (2016)
- [De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers](https://mlsanity.com/paper/1802.08667/) (2018)
- [Augmented Minimax Linear Estimation](https://mlsanity.com/paper/1712.00038/) (2017)
- [Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression](https://mlsanity.com/paper/2210.02002/) (2022)
- [How do noise tails impact on deep ReLU networks?](https://mlsanity.com/paper/2203.10418/) (2022)
- [On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces](https://mlsanity.com/paper/1905.09195/) (2019)
- [Augmented balancing weights as linear regression](https://mlsanity.com/paper/2304.14545/) (2023)
- [Robust Estimation of Causal Effects via High-Dimensional Covariate Balancing Propensity Score](https://mlsanity.com/paper/1812.08683/) (2018)
- [De-Biasing The Lasso With Degrees-of-Freedom Adjustment](https://mlsanity.com/paper/1902.08885/) (2019)
- [The Fundamental Limits of Structure-Agnostic Functional Estimation](https://mlsanity.com/paper/2305.04116/) (2023)
- [Debiased Inverse Propensity Score Weighting for Estimation of Average Treatment Effects with High-Dimensional Confounders](https://mlsanity.com/paper/2011.08661/) (2020)
- [Doubly robust inference via calibration](https://mlsanity.com/paper/2411.02771/) (2024)
- [Causality Pursuit from Heterogeneous Environments via Neural Adversarial Invariance Learning](https://mlsanity.com/paper/2405.04715/) (2024)
- [Doubly-robust inference and optimality in structure-agnostic models with smoothness](https://mlsanity.com/paper/2405.08525/) (2024)
- [Optimal Covariate Balancing Conditions in Propensity Score Estimation](https://mlsanity.com/paper/2108.01255/) (2021)
- [Structure-agnostic Optimality of Doubly Robust Learning for Treatment Effect Estimation](https://mlsanity.com/paper/2402.14264/) (2024)
- [It's Hard to Be Normal: The Impact of Noise on Structure-agnostic Estimation](https://mlsanity.com/paper/2507.02275/) (2025)
- [Challenges of the inconsistency regime: Novel debiasing methods for missing data models](https://mlsanity.com/paper/2309.01362/) (2023)
- [Testability of high-dimensional linear models with non-sparse structures](https://mlsanity.com/paper/1802.09117/) (2018)
- [Sharp Structure-Agnostic Lower Bounds for General Linear Functional Estimation](https://mlsanity.com/paper/2512.17341/) (2025)

## Built on by (most cited first, 3 of 3)

- [On the Asymptotic Inadmissibility of Double Machine Learning Estimators Under Structure-Agnostic Models](https://mlsanity.com/paper/2606.22391/) (2026)
- [Optimal use of a black-box learner in semiparametric estimation](https://mlsanity.com/paper/2607.21541/) (2026)
- [Sharp Structure-Agnostic Minimax Risk for Partial Linear Models](https://mlsanity.com/paper/2609.07997/) (2026)

## Abstract (arXiv)

Modern semiparametric estimation often relies on flexible black-box machine learning methods to estimate nuisance functions, raising a fundamental question: how do nuisance estimation errors propagate into inference for low-dimensional target parameters? The dominant paradigm, exemplified by double machine learning (DML), yields error bounds in which nuisance estimation errors enter multiplicatively. While widely adopted, it remains unclear whether this multiplicative-rate dependence is optimal for black-box models. In this paper, we start by revisiting the partial linear model Y = μ0(X)+T·β0+varepsilon under a structure-agnostic setting, where the nuisance function μ0 is estimated using a generic machine learning model, with approximation error δaμ and stochastic error δμs. We show that the standard DML rate is not optimal in the regime where the auxiliary function mathbbE[T|X=x] cannot be consistently estimated. We propose a new estimator for β0 that achieves a sharper rate of n-1/2+δaμ+(δμs)2 and establish a matching lower bound demonstrating its optimality. Our results reveal a new principle: the first-order stochastic error of nuisance estimation can be eliminated without imposing any additional assumptions. This also leads to a revised tuning strategy favoring under-smoothing, where δaμasymp(δμs)2, rather than the classical bias-variance trade-off δaμ asymp δμs. Under mild additional conditions, the estimator is asymptotically normal with minimal asymptotic variance. The proposed method extends to a broad class of semi-parametric linear functional estimation problems, including average treatment effect estimation. Our results imply that popular orthogonal score methods in semiparametric estimation with black-box nuisance learners can be substantially improved.