Optimal use of a black-box learner in semiparametric estimation

Consider the partial linear model Y = μ0(X) + β0 · T + varepsilon and T = π0(X) + u in the structure-agnostic setting, where we are blind to the structure μ0 and π0 and estimate the nuisances by a black-box hypothesis class. The learnability of the class is characterized by the estimation error δs in the absence of model misspecification and its L2 mis-specification error δa, μ and δa, π for μ0 and π0, respectively. We propose a novel estimator of the target linear coefficient θ0 = β0 with error rate [ frac1√n + δa, μ · δa, π + [δs]2. ] A matching lower bound is also established, implying that this rate is unimprovable. Compared with the product rate yielded by double machine learning (DML), our estimator removes the suboptimal term max(δa, μ, δa, π)· δs at no extra cost or assumption. Building on the underlying insights, which are neither tailored to the one-learner setting nor the partial linear model, we propose Transductive Adversarial Moment-calibrated Editing (TAME), which locally edits debiasing weights induced by black-box regression estimates on the inference sample through adversarial conditional moment calibration. TAME can be combined with any initial black-box estimates and can strictly improve on DML guarantees when the nuisance difficulties are imbalanced. We discuss how to fully exploit the advantages introduced by TAME, including the gains from using two learners, the resulting under-smoothing principle for model selection, and extensions to other linear functional estimation problems.