# Fast Instrument Learning with Faster Rates

arXiv: https://arxiv.org/abs/2205.10772 · HTML: https://mlsanity.com/paper/2205.10772/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 102 references that are in the map.
- First posted on arXiv 22 May 2022.

## Builds on (most cited first, 26 of 26)

- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning](https://mlsanity.com/paper/1412.6614/) (2014)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [User-friendly introduction to PAC-Bayes bounds](https://mlsanity.com/paper/2110.11216/) (2021)
- [Few-Shot Learning via Learning the Representation, Provably](https://mlsanity.com/paper/2002.09434/) (2020)
- [Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences](https://mlsanity.com/paper/1807.02582/) (2018)
- [On the Theory of Transfer Learning: The Importance of Task Diversity](https://mlsanity.com/paper/2006.11650/) (2020)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Orthogonal Statistical Learning](https://mlsanity.com/paper/1901.09036/) (2019)
- [Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel](https://mlsanity.com/paper/1810.05369/) (2018)
- [Kernel Instrumental Variable Regression](https://mlsanity.com/paper/1906.00232/) (2019)
- [Sobolev Norm Learning Rates for Regularized Least-Squares Algorithm](https://mlsanity.com/paper/1702.07254/) (2017)
- [Less is More: Nystr"om Computational Regularization](https://mlsanity.com/paper/1507.04717/) (2015)
- [Deep Generalized Method of Moments for Instrumental Variable Analysis](https://mlsanity.com/paper/1905.12495/) (2019)
- [Limitations of Lazy Training of Two-layers Neural Networks](https://mlsanity.com/paper/1906.08899/) (2019)
- [Minimax Estimation of Conditional Moment Models](https://mlsanity.com/paper/2006.07201/) (2020)
- [Learning Deep Features in Instrumental Variable Regression](https://mlsanity.com/paper/2010.07154/) (2020)
- [Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach](https://mlsanity.com/paper/2007.01290/) (2020)
- [Optimal Sup-norm Rates and Uniform Inference on Nonlinear Functionals of Nonparametric IV Regression](https://mlsanity.com/paper/1508.03365/) (2015)
- [Machine Learning Estimation of Heterogeneous Treatment Effects with Instruments](https://mlsanity.com/paper/1905.10176/) (2019)
- [Estimation and Inference with Trees and Forests in High Dimensions](https://mlsanity.com/paper/2007.03210/) (2020)
- [Regularization Methods for High-Dimensional Instrumental Variables Regression With an Application to Genetical Genomics](https://mlsanity.com/paper/1304.7829/) (2013)
- [Bayesian Deconditional Kernel Mean Embeddings](https://mlsanity.com/paper/1906.00199/) (2019)
- [Mostly Harmless Machine Learning: Learning Optimal Instruments in Linear IV Models](https://mlsanity.com/paper/2011.06158/) (2020)
- [Quasi-Bayesian Dual Instrumental Variable Regression](https://mlsanity.com/paper/2106.08750/) (2021)
- [Instrument Space Selection for Kernel Maximum Moment Restriction](https://mlsanity.com/paper/2106.03340/) (2021)

## Built on by (most cited first, 6 of 6)

- [Spectral Representation Learning for Conditional Moment Models](https://mlsanity.com/paper/2210.16525/) (2022)
- [Koopman-Equivariant Gaussian Processes](https://mlsanity.com/paper/2502.06645/) (2025)
- [Demystifying Spectral Feature Learning for Instrumental Variable Regression](https://mlsanity.com/paper/2506.10899/) (2025)
- [On Uncertainty Quantification for Near-Bayes Optimal Algorithms](https://mlsanity.com/paper/2403.19381/) (2024)
- [Outcome-Aware Spectral Feature Learning for Instrumental Variable Regression](https://mlsanity.com/paper/2512.00919/) (2025)
- [BGM-IV: AI-Powered Bayesian Generative Modeling for Instrumental Variable Regression with High-Dimensional Covariates](https://mlsanity.com/paper/2605.07029/) (2026)

## Abstract (arXiv)

We investigate nonlinear instrumental variable (IV) regression given high-dimensional instruments. We propose a simple algorithm which combines kernelized IV methods and an arbitrary, adaptive regression algorithm, accessed as a black box. Our algorithm enjoys faster-rate convergence and adapts to the dimensionality of informative latent features, while avoiding an expensive minimax optimization procedure, which has been necessary to establish similar guarantees. It further brings the benefit of flexible machine learning models to quasi-Bayesian uncertainty quantification, likelihood-based model selection, and model averaging. Simulation studies demonstrate the competitive performance of our method.