# Challenges in Obtaining Valid Causal Effect Estimates with Machine Learning Algorithms

arXiv: https://arxiv.org/abs/1711.07137 · HTML: https://mlsanity.com/paper/1711.07137/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2017.

## Built on by (most cited first, 9 of 9)

- [Double Machine Learning Methods for Estimating Average Treatment Effects: A Comparative Study](https://mlsanity.com/paper/2204.10969/) (2022)
- [Meta-Learners for Estimation of Causal Effects: Finite Sample Cross-Fit Performance](https://mlsanity.com/paper/2201.12692/) (2022)
- [How to select predictive models for causal inference?](https://mlsanity.com/paper/2302.00370/) (2023)
- [A flexible Bayesian g-formula for causal survival analyses with time-dependent confounding](https://mlsanity.com/paper/2402.02306/) (2024)
- [Performance of Cross-Validated Targeted Maximum Likelihood Estimation](https://mlsanity.com/paper/2409.11265/) (2024)
- [Three Routes to One Answer: Reconciling AIPW, TMLE, and Double Machine Learning for Applied Researchers](https://mlsanity.com/paper/2609.26142/) (2026)
- [Fast Uncertainty Quantification for Kernel-Based Estimators in Large-Scale Causal Inference](https://mlsanity.com/paper/2603.13662/) (2026)
- [When Prediction Error Is Not Enough: Evaluating Nuisance-Function Prediction for Causal Estimation](https://mlsanity.com/paper/2609.00071/) (2026)
- [When Sample-Splitting Choices Matter in TMLE and AIPW: Cross-Fitting, Targeting, and Repetition](https://mlsanity.com/paper/2610.16192/) (2026)

## Abstract (arXiv)

Unlike parametric regression, machine learning (ML) methods do not generally require precise knowledge of the true data generating mechanisms. As such, numerous authors have advocated for ML methods to estimate causal effects. Unfortunately, ML algorithms can perform worse than parametric regression. We demonstrate the performance of ML-based single- and double-robust estimators. We use 100 Monte Carlo samples with sample sizes of 200, 1200, and 5000 to investigate bias and confidence interval coverage under several scenarios. In a simple confounding scenario, confounders were related to the treatment and the outcome via parametric models. In a complex confounding scenario, the simple confounders were transformed to induce complicated nonlinear relationships. In the simple scenario, when ML algorithms were used, double-robust estimators were superior to single-robust estimators. In the complex scenario, single-robust estimators with ML algorithms were at least as biased as estimators using misspecified parametric models. Double-robust estimators were less biased, but coverage was well below nominal. The use of sample splitting, inclusion of confounder interactions, reliance on a richly specified ML algorithm, and use of doubly robust estimators was the only explored approach that yielded negligible bias and nominal coverage. Our results suggest that ML based singly robust methods should be avoided.