# Orthogonal Statistical Learning

arXiv: https://arxiv.org/abs/1901.09036 · HTML: https://mlsanity.com/paper/1901.09036/

## Key facts

- Cited by 151 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jan 2019.

## Built on by (most cited first, 30 of 151)

- [A Review of Off-Policy Evaluation in Reinforcement Learning](https://mlsanity.com/paper/2212.06355/) (2022)
- [On Inductive Biases for Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2106.03765/) (2021)
- [Causal machine learning for predicting treatment outcomes](https://mlsanity.com/paper/2410.08770/) (2024)
- [Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiency](https://mlsanity.com/paper/2102.02981/) (2021)
- [Causally motivated Shortcut Removal Using Auxiliary Labels](https://mlsanity.com/paper/2105.06422/) (2021)
- [CausalPFN: Amortized Causal Effect Estimation via In-Context Learning](https://mlsanity.com/paper/2506.07918/) (2025)
- [Causal Inference Under Unmeasured Confounding With Negative Controls: A Minimax Learning Approach](https://mlsanity.com/paper/2103.14029/) (2021)
- [Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning](https://mlsanity.com/paper/2202.09667/) (2022)
- [Foundation Models for Causal Inference via Prior-Data Fitted Networks](https://mlsanity.com/paper/2506.10914/) (2025)
- [Applied Causal Inference Powered by ML and AI](https://mlsanity.com/paper/2403.02467/) (2024)
- [Knowledge Distillation as Semiparametric Inference](https://mlsanity.com/paper/2104.09732/) (2021)
- [Estimating average causal effects from patient trajectories](https://mlsanity.com/paper/2203.01228/) (2022)
- [Doubly-Valid/Doubly-Sharp Sensitivity Analysis for Causal Inference with Unmeasured Confounding](https://mlsanity.com/paper/2112.11449/) (2021)
- [B-Learner: Quasi-Oracle Bounds on Heterogeneous Causal Effects Under Hidden Confounding](https://mlsanity.com/paper/2304.10577/) (2023)
- [Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2205.13589/) (2022)
- [Exploring Transformer Backbones for Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2202.01336/) (2022)
- [Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation](https://mlsanity.com/paper/2211.01939/) (2022)
- [Future-Dependent Value-Based Off-Policy Evaluation in POMDPs](https://mlsanity.com/paper/2207.13081/) (2022)
- [Minimax Kernel Machine Learning for a Class of Doubly Robust Functionals with Application to Proximal Causal Inference](https://mlsanity.com/paper/2104.02929/) (2021)
- [Model-agnostic meta-learners for estimating heterogeneous treatment effects over time](https://mlsanity.com/paper/2407.05287/) (2024)
- [Robust and Agnostic Learning of Conditional Distributional Treatment Effects](https://mlsanity.com/paper/2205.11486/) (2022)
- [Off-Policy Evaluation for Episodic Partially Observable Markov Decision Processes under Non-Parametric Models](https://mlsanity.com/paper/2209.10064/) (2022)
- [Doubly Robust Self-Training](https://mlsanity.com/paper/2306.00265/) (2023)
- [DiffPO: A causal diffusion model for learning distributions of potential outcomes](https://mlsanity.com/paper/2410.08924/) (2024)
- [Causal isotonic calibration for heterogeneous treatment effects](https://mlsanity.com/paper/2302.14011/) (2023)
- [Reliable Off-Policy Learning for Dosage Combinations](https://mlsanity.com/paper/2305.19742/) (2023)
- [Differentially Private Estimation of Heterogeneous Causal Effects](https://mlsanity.com/paper/2202.11043/) (2022)
- [Learning from a Biased Sample](https://mlsanity.com/paper/2209.01754/) (2022)
- [TabPFN: One Model to Rule Them All?](https://mlsanity.com/paper/2505.20003/) (2025)
- [Robust Fitted-Q-Evaluation and Iteration under Sequentially Exogenous Unobserved Confounders](https://mlsanity.com/paper/2302.00662/) (2023)

## Abstract (arXiv)

We provide non-asymptotic excess risk guarantees for statistical learning in a setting where the population risk with respect to which we evaluate the target parameter depends on an unknown nuisance parameter that must be estimated from data. We analyze a two-stage sample splitting meta-algorithm that takes as input arbitrary estimation algorithms for the target parameter and nuisance parameter. We show that if the population risk satisfies a condition called Neyman orthogonality, the impact of the nuisance estimation error on the excess risk bound achieved by the meta-algorithm is of second order. Our theorem is agnostic to the particular algorithms used for the target and nuisance and only makes an assumption on their individual performance. This enables the use of a plethora of existing results from machine learning to give new guarantees for learning with a nuisance component. Moreover, by focusing on excess risk rather than parameter estimation, we can provide rates under weaker assumptions than in previous works and accommodate settings in which the target parameter belongs to a complex nonparametric class. We provide conditions on the metric entropy of the nuisance and target classes such that oracle rates of the same order as if we knew the nuisance parameter are achieved.