# Debiased Machine Learning without Sample-Splitting for Stable Estimators

arXiv: https://arxiv.org/abs/2206.01825 · HTML: https://mlsanity.com/paper/2206.01825/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 67 references that are in the map.
- First posted on arXiv 3 Jun 2022.

## Builds on (most cited first, 19 of 19)

- [Train faster, generalize better: Stability of stochastic gradient descent](https://mlsanity.com/paper/1509.01240/) (2015)
- [De-biasing the Lasso: Optimal Sample Size for Gaussian Designs](https://mlsanity.com/paper/1508.02757/) (2015)
- [Adversarial Estimation of Riesz Representers](https://mlsanity.com/paper/2101.00009/) (2020)
- [De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers](https://mlsanity.com/paper/1802.08667/) (2018)
- [Cross-validation Confidence Intervals for Test Error](https://mlsanity.com/paper/2007.12671/) (2020)
- [Program Evaluation and Causal Inference with High-Dimensional Data](https://mlsanity.com/paper/1311.2645/) (2013)
- [A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models](https://mlsanity.com/paper/1412.8765/) (2014)
- [Hypothesis Testing in High-Dimensional Regression under the Gaussian Random Design Model: Asymptotic Theory](https://mlsanity.com/paper/1301.4240/) (2013)
- [Asymptotic normality and optimalities in estimation of large Gaussian graphical models](https://mlsanity.com/paper/1309.6024/) (2013)
- [Uniform Post Selection Inference for LAD Regression and Other Z-estimation problems](https://mlsanity.com/paper/1304.0282/) (2013)
- [A Simple and General Debiased Machine Learning Theorem with Finite Sample Guarantees](https://mlsanity.com/paper/2105.15197/) (2021)
- [Linear Hypothesis Testing in Dense High-Dimensional Linear Models](https://mlsanity.com/paper/1610.02987/) (2016)
- [Confidence intervals for high-dimensional inverse covariance estimation](https://mlsanity.com/paper/1403.6752/) (2014)
- [A Unified Theory of Confidence Regions and Testing for High Dimensional Estimating Equations](https://mlsanity.com/paper/1510.08986/) (2015)
- [Semi-parametric efficiency bounds for high-dimensional models](https://mlsanity.com/paper/1601.00815/) (2016)
- [Uniform Inference for High-dimensional Quantile Regression: Linear Functionals and Regression Rank Scores](https://mlsanity.com/paper/1702.06209/) (2017)
- [Non-Parametric Inference Adaptive to Intrinsic Dimension](https://mlsanity.com/paper/1901.03719/) (2019)
- [Confidence regions for high-dimensional generalized linear models under sparsity](https://mlsanity.com/paper/1610.01353/) (2016)
- [Breaking the curse of dimensionality in regression](https://mlsanity.com/paper/1708.00430/) (2017)

## Built on by (most cited first, 13 of 13)

- [Bagging Provides Assumption-free Stability](https://mlsanity.com/paper/2301.12600/) (2023)
- [Robust Fitted-Q-Evaluation and Iteration under Sequentially Exogenous Unobserved Confounders](https://mlsanity.com/paper/2302.00662/) (2023)
- [Structure-agnostic Optimality of Doubly Robust Learning for Treatment Effect Estimation](https://mlsanity.com/paper/2402.14264/) (2024)
- [RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventions](https://mlsanity.com/paper/2312.06403/) (2023)
- [Semi-Supervised U-statistics](https://mlsanity.com/paper/2402.18921/) (2024)
- [Maximally Machine-Learnable Portfolios](https://mlsanity.com/paper/2306.05568/) (2023)
- [FairViT: Fair Vision Transformer via Adaptive Masking](https://mlsanity.com/paper/2407.14799/) (2024)
- [Automatic debiasing of neural networks via moment-constrained learning](https://mlsanity.com/paper/2409.19777/) (2024)
- [Is Algorithmic Stability Testable? A Unified Framework under Computational Constraints](https://mlsanity.com/paper/2405.15107/) (2024)
- [Inference in Partially Linear Models under Dependent Data with Deep Neural Networks](https://mlsanity.com/paper/2410.22574/) (2024)
- [A Meta-Learning Method for Estimation of Causal Excursion Effects to Assess Time-Varying Moderation](https://mlsanity.com/paper/2306.16297/) (2023)
- [Cross-Fitting-Free Debiased Machine Learning with Multiway Dependence](https://mlsanity.com/paper/2602.11333/) (2026)
- [Bootstrap validity in Bayesian semi-parametric models](https://mlsanity.com/paper/2608.06670/) (2026)

## Abstract (arXiv)

Estimation and inference on causal parameters is typically reduced to a generalized method of moments problem, which involves auxiliary functions that correspond to solutions to a regression or classification problem. Recent line of work on debiased machine learning shows how one can use generic machine learning estimators for these auxiliary problems, while maintaining asymptotic normality and root-n consistency of the target parameter of interest, while only requiring mean-squared-error guarantees from the auxiliary estimation algorithms. The literature typically requires that these auxiliary problems are fitted on a separate sample or in a cross-fitting manner. We show that when these auxiliary estimation algorithms satisfy natural leave-one-out stability properties, then sample splitting is not required. This allows for sample re-use, which can be beneficial in moderately sized sample regimes. For instance, we show that the stability properties that we propose are satisfied for ensemble bagged estimators, built via sub-sampling without replacement, a popular technique in machine learning practice.