mlsanityData to 7 Oct 2026

Stochastic Optimization Algorithms for Instrumental Variable Regression with Streaming Data

arXiv 2405.19463 ↗First posted 29 May 2024stat.ML cs.LG econ.EMcited by 4

Not on the map. Why?

Abstract (arXiv)

We develop and analyze algorithms for instrumental variable regression by viewing the problem as a conditional stochastic optimization problem. In the context of least-squares instrumental variable regression, our algorithms neither require matrix inversions nor mini-batches and provides a fully online approach for performing instrumental variable regression with streaming data. When the true model is linear, we derive rates of convergence in expectation, that are of order O(log T/T) and O(1/T1-iota) for any iota>0, respectively under the availability of two-sample and one-sample oracles, respectively, where T is the number of iterations. Importantly, under the availability of the two-sample oracle, our procedure avoids explicitly modeling and estimating the relationship between confounder and the instrumental variables, demonstrating the benefit of the proposed approach over recent works based on reformulating the problem as minimax optimization problems. Numerical experiments are provided to corroborate the theoretical results.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 40 of its 75 references that are in the map. First posted on arXiv 29 May 2024.

Builds on

40 of 75 references.

References · 40

  1. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  2. 2020Better Theory for SGD in the Nonconvex Worldcited by 156
  3. 2019Kernel Instrumental Variable Regressioncited by 118
  4. 2014Stochastic Compositional Gradient Descent: Algorithms for Minimizing Compositions of Expected-Value Functionscited by 93
  5. 2019Deep Generalized Method of Moments for Instrumental Variable Analysiscited by 88
  6. 2020Minimax Estimation of Conditional Moment Modelscited by 83
  7. 2016Statistical Inference for Model Parameters in Stochastic Gradient Descentcited by 71
  8. 2018Random Shuffling Beats SGD after Finite Epochscited by 66
  9. 2020Learning Deep Features in Instrumental Variable Regressioncited by 65
  10. 2018On Markov Chain Gradient Descentcited by 63
  11. 2019SGD without Replacement: Sharper Rates for General Smooth Convex Functionscited by 59
  12. 2020Semiparametric proximal causal inferencecited by 58
28 more
  1. 2020SGD with shuffling: optimal rates without component convexity and large epoch requirementscited by 57
  2. 2020Closing the convergence gap of SGD without replacementcited by 56
  3. 2021Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restrictioncited by 54
  4. 2018A Single Time-Scale Stochastic Approximation Method for Nested Stochastic Optimizationcited by 53
  5. 2020Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimizationcited by 51
  6. 2020Biased Stochastic First-Order Methods for Conditional Stochastic Optimization and Applications in Meta Learningcited by 46
  7. 2020Online Covariance Matrix Estimation in Stochastic Gradient Descentcited by 37
  8. 2023Stochastic Gradient Descent under Markovian Sampling Schemescited by 37
  9. 2020Nonlinear Two-Time-Scale Stochastic Approximation: Convergence and Finite-Time Performancecited by 35
  10. 2018Adversarial Generalized Method of Momentscited by 34
  11. 2020Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approachcited by 31
  12. 2019Normal Approximation for Stochastic Gradient Descent via Non-Asymptotic Rates of Martingale CLTcited by 30
  13. 2019Multi-Level Composite Stochastic Optimization via Nested Variance Reductioncited by 28
  14. 2015Convergence Rate of Incremental Gradient and Newton Methodscited by 26
  15. 2020Stochastic Multi-level Composition Optimization Algorithms with Level-Independent Convergence Ratescited by 25
  16. 2019Sample Complexity of Sample Average Approximation for Conditional Stochastic Optimizationcited by 19
  17. 2020Sample Complexity Bounds for Two Timescale Value-based Reinforcement Learning Algorithmscited by 19
  18. 2020A Stochastic Subgradient Method for Nonsmooth Nonconvex Multi-Level Composition Optimizationcited by 16
  19. 2023Contextual Stochastic Bilevel Optimizationcited by 14
  20. 2023Minimax Instrumental Variable Regression and L2 Convergence Guarantees without Identification or Closednesscited by 13
  21. 2022Causal Inference with Treatment Measurement Error: A Nonparametric Instrumental Variable Approachcited by 9
  22. 2021Non-Asymptotic Analysis for Two Time-scale TDC with General Smooth Function Approximationcited by 8
  23. 2023Stochastic Online Instrumental Variable Regression: Regrets for Endogeneity and Bandit Feedbackcited by 7
  24. 2017Is completeness necessary? Estimation in nonidentified linear modelscited by 6
  25. 2022Constrained Stochastic Nonconvex Optimization with State-dependent Markov Datacited by 6
  26. 2024Nonparametric Instrumental Variable Regression through Stochastic Approximate Gradientscited by 6
  27. 2019Finite-Time Performance of Distributed Two-Time-Scale Stochastic Approximationcited by 3
  28. 2023SGMM: Stochastic Approximation to Generalized Method of Momentscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Transformers Handle Endogeneity in In-Context Linear Regressioncited by 8
  2. 2024Nonparametric Instrumental Regression via Kernel Methods is Minimax Optimalcited by 7
  3. 2025Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regressioncited by 0
  4. 2026Convergence Rate of a Functional Learning Method for Contextual Stochastic Optimizationcited by 0