# Machine learning the first stage in 2SLS: Practical guidance from bias decomposition and simulation

arXiv: https://arxiv.org/abs/2505.13422 · HTML: https://mlsanity.com/paper/2505.13422/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 59 references that are in the map.
- First posted on arXiv 19 May 2025.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Kernel Instrumental Variable Regression](https://mlsanity.com/paper/1906.00232/) (2019)
- [Deep Generalized Method of Moments for Instrumental Variable Analysis](https://mlsanity.com/paper/1905.12495/) (2019)
- [Learning Deep Features in Instrumental Variable Regression](https://mlsanity.com/paper/2010.07154/) (2020)
- [Adversarial Estimation of Riesz Representers](https://mlsanity.com/paper/2101.00009/) (2020)
- [Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach](https://mlsanity.com/paper/2007.01290/) (2020)
- [A Class of Algorithms for General Instrumental Variable Models](https://mlsanity.com/paper/2006.06366/) (2020)
- [On Deep Instrumental Variables Estimate](https://mlsanity.com/paper/2004.14954/) (2020)
- [In Defense of the Indefensible: A Very Naive Approach to High-Dimensional Inference](https://mlsanity.com/paper/1705.05543/) (2017)
- [Mostly Harmless Machine Learning: Learning Optimal Instruments in Linear IV Models](https://mlsanity.com/paper/2011.06158/) (2020)

## Abstract (arXiv)

Machine learning (ML) primarily evolved to solve "prediction problems." The first stage of two-stage least squares (2SLS) is a prediction problem, suggesting potential gains from ML first-stage assistance. However, little guidance exists on when ML helps 2SLSunicodex2014or when it hurts. We investigate the implications of inserting ML into 2SLS, decomposing the bias into three informative components. Mechanically, ML-in-2SLS procedures face issues common to prediction and causal-inference settingsunicodex2014and their interaction. Through simulation, we show linear ML methods (e.g., post-Lasso) work well, while nonlinear methods (e.g., random forests, neural nets) generate substantial bias in second-stage estimatesunicodex2014potentially exceeding the bias of endogenous OLS.