# The Fundamental Limits of Structure-Agnostic Functional Estimation

arXiv: https://arxiv.org/abs/2305.04116 · HTML: https://mlsanity.com/paper/2305.04116/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 73 references that are in the map.
- First posted on arXiv 6 May 2023.

## Builds on (most cited first, 14 of 14)

- [Orthogonal Statistical Learning](https://mlsanity.com/paper/1901.09036/) (2019)
- [Semiparametric doubly robust targeted double machine learning: a review](https://mlsanity.com/paper/2203.06469/) (2022)
- [Plugin Estimation of Smooth Optimal Transport Maps](https://mlsanity.com/paper/2107.12364/) (2021)
- [Efficient multivariate entropy estimation via k-nearest neighbour distances](https://mlsanity.com/paper/1606.00304/) (2016)
- [Semi-supervised Inference for Explained Variance in High-dimensional Linear Regression and Its Applications](https://mlsanity.com/paper/1806.06179/) (2018)
- [Hypothesis Testing For Densities and High-Dimensional Multinomials: Sharp Local Minimax Rates](https://mlsanity.com/paper/1706.10003/) (2017)
- [Higher Order Tangent Spaces and Influence Functions](https://mlsanity.com/paper/1502.00812/) (2015)
- [Nonparametric Estimation of Renyi Divergence and Friends](https://mlsanity.com/paper/1402.2966/) (2014)
- [Rejoinder: On nearly assumption-free tests of nominal confidence interval coverage for causal parameters estimated by machine learning](https://mlsanity.com/paper/2008.03288/) (2020)
- [When is the estimated propensity score better? High-dimensional analysis and bias correction](https://mlsanity.com/paper/2303.17102/) (2023)
- [On Doubly Robust Inference for Double Machine Learning in Semiparametric Regression](https://mlsanity.com/paper/2107.06124/) (2021)
- [Influence Functions for Machine Learning: Nonparametric Estimators for Entropies, Divergences and Mutual Informations](https://mlsanity.com/paper/1411.4342/) (2014)
- [Efficient functional estimation and the super-oracle phenomenon](https://mlsanity.com/paper/1904.09347/) (2019)
- [Minimax Estimation of Quadratic Fourier Functionals](https://mlsanity.com/paper/1803.11451/) (2018)

## Built on by (most cited first, 12 of 12)

- [Structure-agnostic Optimality of Doubly Robust Learning for Treatment Effect Estimation](https://mlsanity.com/paper/2402.14264/) (2024)
- [Efficient and Multiply Robust Risk Estimation under General Forms of Dataset Shift](https://mlsanity.com/paper/2306.16406/) (2023)
- [Orthogonal Representation Learning for Estimating Causal Quantities](https://mlsanity.com/paper/2502.04274/) (2025)
- [Automated Efficient Estimation using Monte Carlo Efficient Influence Functions](https://mlsanity.com/paper/2403.00158/) (2024)
- [Optimally taming biases in black-box models for efficient semiparametric estimation](https://mlsanity.com/paper/2606.06368/) (2026)
- [On the Asymptotic Inadmissibility of Double Machine Learning Estimators Under Structure-Agnostic Models](https://mlsanity.com/paper/2606.22391/) (2026)
- [Assumption-lean falsification tests of rate double-robustness of double-machine-learning estimators](https://mlsanity.com/paper/2306.10590/) (2023)
- [Covariance alignment: from maximum likelihood estimation to Gromov-Wasserstein](https://mlsanity.com/paper/2311.13595/) (2023)
- [Causal Inference for Genomic Data with Multiple Heterogeneous Outcomes](https://mlsanity.com/paper/2404.09119/) (2024)
- [GDR-learners: Orthogonal Learning of Generative Models for Potential Outcomes](https://mlsanity.com/paper/2509.22953/) (2025)
- [Optimal use of a black-box learner in semiparametric estimation](https://mlsanity.com/paper/2607.21541/) (2026)
- [Sharp Structure-Agnostic Minimax Risk for Partial Linear Models](https://mlsanity.com/paper/2609.07997/) (2026)

## Abstract (arXiv)

Many recent developments in causal inference, and functional estimation problems more generally, have been motivated by the fact that classical one-step (first-order) debiasing methods, or their more recent sample-split double machine-learning avatars, can outperform plugin estimators under surprisingly weak conditions. These first-order corrections improve on plugin estimators in a black-box fashion, and consequently are often used in conjunction with powerful off-the-shelf estimation methods. These first-order methods are however provably suboptimal in a minimax sense for functional estimation when the nuisance functions live in Holder-type function spaces. This suboptimality of first-order debiasing has motivated the development of "higher-order" debiasing methods. The resulting estimators are, in some cases, provably optimal over Holder-type spaces, but both the estimators which are minimax-optimal and their analyses are crucially tied to properties of the underlying function space. In this paper we investigate the fundamental limits of structure-agnostic functional estimation, where relatively weak conditions are placed on the underlying nuisance functions. We show that there is a strong sense in which existing first-order methods are optimal. We achieve this goal by providing a formalization of the problem of functional estimation with black-box nuisance function estimates, and deriving minimax lower bounds for this problem. Our results highlight some clear tradeoffs in functional estimation -- if we wish to remain agnostic to the underlying nuisance function spaces, impose only high-level rate conditions, and maintain compatibility with black-box nuisance estimators then first-order methods are optimal. When we have an understanding of the structure of the underlying nuisance functions then carefully constructed higher-order estimators can outperform first-order estimators.