mlsanityData to 13 Oct 2026

Nonparametric Instrumental Regression via Kernel Methods is Minimax Optimal

arXiv 2411.19653 ↗First posted 29 Nov 2024stat.ML cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

We study the kernel instrumental variable (KIV) algorithm, a kernel-based two-stage least-squares method for nonparametric instrumental variable regression. We provide a convergence analysis covering both identified and non-identified regimes: when the structural function is not identified, we show that the KIV estimator converges to the minimum-norm IV solution in the reproducing kernel Hilbert space associated with the kernel. Crucially, we establish convergence in the strong L2 norm, rather than only in a pseudo-norm. We quantify statistical difficulty through a link condition that compares the covariance structure of the endogenous regressor with that induced by the instrument, yielding an interpretable measure of ill-posedness. Under standard eigenvalue-decay and source assumptions, we derive strong L2 learning rates for KIV and prove that they are minimax-optimal over fixed smoothness classes. Finally, we replace the stage-1 Tikhonov step by general spectral regularization, thereby avoiding saturation and improving rates for smoother first-stage targets. The matching lower bound shows that instrumental regression induces an unavoidable slowdown relative to ordinary kernel ridge regression.

Key facts. Cited by 7 papers in the map (data through 13 Oct 2026). Builds on 34 of its 84 references that are in the map. First posted on arXiv 29 Nov 2024.

Builds on

34 of 84 references.

References · 34

  1. 2019Kernel Instrumental Variable Regressioncited by 118
  2. 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 113
  3. 2015Less is More: Nystr"om Computational Regularizationcited by 90
  4. 2019Deep Generalized Method of Moments for Instrumental Variable Analysiscited by 88
  5. 2020A Measure-Theoretic Approach to Kernel Conditional Mean Embeddingscited by 86
  6. 2020Minimax Estimation of Conditional Moment Modelscited by 83
  7. 2022Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spacescited by 67
  8. 2020Learning Deep Features in Instrumental Variable Regressioncited by 65
  9. 2018Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spacescited by 59
  10. 2021Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiencycited by 56
  11. 2021Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restrictioncited by 55
  12. 2022Optimal Rates for Regularized Conditional Mean Embedding Learningcited by 50
22 more
  1. 2023Sharp Spectral Rates for Koopman Operator Learningcited by 36
  2. 2020A General Framework for Consistent Structured Prediction with Implicit Loss Embeddingscited by 35
  3. 2021Instrumental Variable Value Iteration for Causal Offline Reinforcement Learningcited by 35
  4. 2016A Consistent Regularization Approach for Structured Predictioncited by 33
  5. 2019A Rigorous Theory of Conditional Mean Embeddingscited by 31
  6. 2020Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approachcited by 31
  7. 2015Optimal Sup-norm Rates and Uniform Inference on Nonlinear Functionals of Nonparametric IV Regressioncited by 30
  8. 2022Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problemcited by 28
  9. 2021On Instrumental Variable Regression for Deep Offline Policy Evaluationcited by 22
  10. 2023Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithmcited by 22
  11. 2019Beating SGD Saturation with Tail-Averaging and Minibatchingcited by 21
  12. 2020Nonparametric approximation of conditional expectation operatorscited by 21
  13. 2024Functional Bilevel Optimization for Machine Learningcited by 20
  14. 2020Instrumental Variable Regression via Kernel Maximum Moment Losscited by 14
  15. 2023Minimax Instrumental Variable Regression and L2 Convergence Guarantees without Identification or Closednesscited by 13
  16. 2024Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithmscited by 12
  17. 2024Spectral Representation for Causal Estimation with Hidden Confounderscited by 12
  18. 2016Semiparametric Estimation with Data Missing Not at Random Using an Instrumental Variablecited by 9
  19. 2023Source Condition Double Robust Inference on Functionals of Inverse Problemscited by 9
  20. 2017Is completeness necessary? Estimation in nonidentified linear modelscited by 6
  21. 2024Regularized DeepIV with Model Selectioncited by 6
  22. 2024Stochastic Optimization Algorithms for Instrumental Variable Regression with Streaming Datacited by 4

Built on by

Cited by year

Citing papers per year: 2025: 7.

7 papers.

  1. 2025Nested Expectations with Kernel Quadraturecited by 5
  2. 2025Demystifying Spectral Feature Learning for Instrumental Variable Regressioncited by 4
  3. 2025Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regressioncited by 3
  4. 2025Regularized least squares learning with heavy-tailed noise is minimax optimalcited by 3
  5. 2025Learning Theory for Kernel Bilevel Optimizationcited by 2
  6. 2025Outcome-Aware Spectral Feature Learning for Instrumental Variable Regressioncited by 2
  7. 2025Density Ratio-Free Doubly Robust Proxy Causal Learningcited by 1