# Adaptive Instrument Design for Indirect Experiments

arXiv: https://arxiv.org/abs/2312.02438 · HTML: https://mlsanity.com/paper/2312.02438/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 94 references that are in the map.
- First posted on arXiv 5 Dec 2023.

## Builds on (most cited first, 30 of 38)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Studying Large Language Model Generalization with Influence Functions](https://mlsanity.com/paper/2308.03296/) (2023)
- [Optimizing Millions of Hyperparameters by Implicit Differentiation](https://mlsanity.com/paper/1911.02590/) (2019)
- [Influence Functions in Deep Learning Are Fragile](https://mlsanity.com/paper/2006.14651/) (2020)
- [If Influence Functions are the Answer, Then What is the Question?](https://mlsanity.com/paper/2209.05364/) (2022)
- [Scaling Up Influence Functions](https://mlsanity.com/paper/2112.03052/) (2021)
- [Modern Bayesian Experimental Design](https://mlsanity.com/paper/2302.14545/) (2023)
- [VarGrad: A Low-Variance Gradient Estimator for Variational Inference](https://mlsanity.com/paper/2010.10436/) (2020)
- [Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines](https://mlsanity.com/paper/1803.07246/) (2018)
- [Learning Deep Features in Instrumental Variable Regression](https://mlsanity.com/paper/2010.07154/) (2020)
- [Dataset Distillation with Convexified Implicit Gradients](https://mlsanity.com/paper/2302.06755/) (2023)
- [Statistical Inference with M-Estimators on Adaptively Collected Data](https://mlsanity.com/paper/2104.14074/) (2021)
- [The Influence Function of Semiparametric Estimators](https://mlsanity.com/paper/1508.01378/) (2015)
- [Discriminative Jackknife: Quantifying Uncertainty in Deep Learning via Higher-Order Influence Functions](https://mlsanity.com/paper/2007.13481/) (2020)
- [Machine Learning Estimation of Heterogeneous Treatment Effects with Instruments](https://mlsanity.com/paper/1905.10176/) (2019)
- [Efficient Adaptive Experimental Design for Average Treatment Effect Estimation](https://mlsanity.com/paper/2002.05308/) (2020)
- [Gradient Estimation with Discrete Stein Operators](https://mlsanity.com/paper/2202.09497/) (2022)
- [Helping the Helper: Supporting Peer Counselors via AI-Empowered Practice and Feedback](https://mlsanity.com/paper/2305.08982/) (2023)
- [Visually Communicating and Teaching Intuition for Influence Functions](https://mlsanity.com/paper/1810.03260/) (2018)
- [Auto IV: Counterfactual Prediction via Automatic Instrumental Variable Decomposition](https://mlsanity.com/paper/2107.05884/) (2021)
- [Estimating individual treatment effects under unobserved confounding using binary instruments](https://mlsanity.com/paper/2208.08544/) (2022)
- [Instrumental Variables: An Econometrician's Perspective](https://mlsanity.com/paper/1410.0163/) (2014)
- [Accounting For Informative Sampling When Learning to Forecast Treatment Outcomes Over Time](https://mlsanity.com/paper/2306.04255/) (2023)
- [Optimal Experimental Design for Staggered Rollouts](https://mlsanity.com/paper/1911.03764/) (2019)
- [Efficient Online Estimation of Causal Effects by Deciding What to Observe](https://mlsanity.com/paper/2108.09265/) (2021)
- [Stability revisited: new generalisation bounds for the Leave-one-Out](https://mlsanity.com/paper/1608.06412/) (2016)
- [Instrument-Armed Bandits](https://mlsanity.com/paper/1705.07377/) (2017)
- [Instrumental Variables in Causal Inference and Machine Learning: A Survey](https://mlsanity.com/paper/2212.05778/) (2022)

## Built on by (most cited first, 6 of 6)

- [Active Statistical Inference](https://mlsanity.com/paper/2403.03208/) (2024)
- [Targeted Sequential Indirect Experiment Design](https://mlsanity.com/paper/2405.19985/) (2024)
- [Efficient Adaptive Experimentation with Noncompliance](https://mlsanity.com/paper/2505.17468/) (2025)
- [Generalized Encouragement-Based Instrumental Variables for Counterfactual Regression](https://mlsanity.com/paper/2408.05428/) (2024)
- [Online Data Collection for Efficient Semiparametric Inference](https://mlsanity.com/paper/2411.03195/) (2024)
- [Gaussian Process Bandits for AI System Optimization with Selectively Verified Feedback](https://mlsanity.com/paper/2610.17760/) (2026)

## Abstract (arXiv)

Indirect experiments provide a valuable framework for estimating treatment effects in situations where conducting randomized control trials (RCTs) is impractical or unethical. Unlike RCTs, indirect experiments estimate treatment effects by leveraging (conditional) instrumental variables, enabling estimation through encouragement and recommendation rather than strict treatment assignment. However, the sample efficiency of such estimators depends not only on the inherent variability in outcomes but also on the varying compliance levels of users with the instrumental variables and the choice of estimator being used, especially when dealing with numerous instrumental variables. While adaptive experiment design has a rich literature for direct experiments, in this paper we take the initial steps towards enhancing sample efficiency for indirect experiments by adaptively designing a data collection policy over instrumental variables. Our main contribution is a practical computational procedure that utilizes influence functions to search for an optimal data collection policy, minimizing the mean-squared error of the desired (non-linear) estimator. Through experiments conducted in various domains inspired by real-world applications, we showcase how our method can significantly improve the sample efficiency of indirect experiments.