mlsanityData to 7 Oct 2026

Adaptive Instrument Design for Indirect Experiments

arXiv 2312.02438 ↗First posted 5 Dec 2023cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Indirect experiments provide a valuable framework for estimating treatment effects in situations where conducting randomized control trials (RCTs) is impractical or unethical. Unlike RCTs, indirect experiments estimate treatment effects by leveraging (conditional) instrumental variables, enabling estimation through encouragement and recommendation rather than strict treatment assignment. However, the sample efficiency of such estimators depends not only on the inherent variability in outcomes but also on the varying compliance levels of users with the instrumental variables and the choice of estimator being used, especially when dealing with numerous instrumental variables. While adaptive experiment design has a rich literature for direct experiments, in this paper we take the initial steps towards enhancing sample efficiency for indirect experiments by adaptively designing a data collection policy over instrumental variables. Our main contribution is a practical computational procedure that utilizes influence functions to search for an optimal data collection policy, minimizing the mean-squared error of the desired (non-linear) estimator. Through experiments conducted in various domains inspired by real-world applications, we showcase how our method can significantly improve the sample efficiency of indirect experiments.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 38 of its 94 references that are in the map. First posted on arXiv 5 Dec 2023.

Builds on

38 of 94 references.

References · 38

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  3. 2015Estimation and Inference of Heterogeneous Treatment Effects using Random Forestscited by 597
  4. 2023Studying Large Language Model Generalization with Influence Functionscited by 311
  5. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  6. 2020Influence Functions in Deep Learning Are Fragilecited by 247
  7. 2022If Influence Functions are the Answer, Then What is the Question?cited by 142
  8. 2021Scaling Up Influence Functionscited by 130
  9. 2023Modern Bayesian Experimental Designcited by 121
  10. 2020VarGrad: A Low-Variance Gradient Estimator for Variational Inferencecited by 73
  11. 2018Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselinescited by 65
  12. 2020Learning Deep Features in Instrumental Variable Regressioncited by 65
26 more
  1. 2023Dataset Distillation with Convexified Implicit Gradientscited by 46
  2. 2021Statistical Inference with M-Estimators on Adaptively Collected Datacited by 40
  3. 2015The Influence Function of Semiparametric Estimatorscited by 36
  4. 2020Discriminative Jackknife: Quantifying Uncertainty in Deep Learning via Higher-Order Influence Functionscited by 34
  5. 2019Machine Learning Estimation of Heterogeneous Treatment Effects with Instrumentscited by 29
  6. 2023Helping the Helper: Supporting Peer Counselors via AI-Empowered Practice and Feedbackcited by 23
  7. 2020Efficient Adaptive Experimental Design for Average Treatment Effect Estimationcited by 22
  8. 2022Gradient Estimation with Discrete Stein Operatorscited by 22
  9. 2018Visually Communicating and Teaching Intuition for Influence Functionscited by 19
  10. 2021Auto IV: Counterfactual Prediction via Automatic Instrumental Variable Decompositioncited by 18
  11. 2022Estimating individual treatment effects under unobserved confounding using binary instrumentscited by 17
  12. 2014Instrumental Variables: An Econometrician's Perspectivecited by 15
  13. 2023Accounting For Informative Sampling When Learning to Forecast Treatment Outcomes Over Timecited by 14
  14. 2019Optimal Experimental Design for Staggered Rolloutscited by 13
  15. 2021Efficient Online Estimation of Causal Effects by Deciding What to Observecited by 11
  16. 2016Stability revisited: new generalisation bounds for the Leave-one-Outcited by 10
  17. 2017Instrument-Armed Banditscited by 10
  18. 2022Instrumental Variables in Causal Inference and Machine Learning: A Surveycited by 10
  19. 2021Double Control Variates for Gradient Estimation in Discrete Latent Variable Modelscited by 8
  20. 2019An Exponential Efron-Stein Inequality for Lq Stable Learning Rulescited by 6
  21. 2021Lifelong Hyper-Policy Optimization with Multiple Importance Sampling Regularizationcited by 5
  22. 2021Biased Gradient Estimate with Drastic Variance Reduction for Meta Reinforcement Learningcited by 5
  23. 2021Incentivizing Compliance with Algorithmic Instrumentscited by 4
  24. 2023Double machine learning and design in batch adaptive experimentscited by 4
  25. 2023ACE: Active Learning for Causal Inference with Expensive Experimentscited by 3
  26. 2020Estimation of local treatment effects under the binary instrumental variable modelcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 1.

5 papers.

  1. 2024Active Statistical Inferencecited by 46
  2. 2024Targeted Sequential Indirect Experiment Designcited by 4
  3. 2025Efficient Adaptive Experimentation with Noncompliancecited by 2
  4. 2024Generalized Encouragement-Based Instrumental Variables for Counterfactual Regressioncited by 0
  5. 2024Online Data Collection for Efficient Semiparametric Inferencecited by 0