# Demystifying Spectral Feature Learning for Instrumental Variable Regression

arXiv: https://arxiv.org/abs/2506.10899 · HTML: https://mlsanity.com/paper/2506.10899/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Builds on (most cited first, 26 of 26)

- [Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss](https://mlsanity.com/paper/2106.04156/) (2021)
- [Kernel Instrumental Variable Regression](https://mlsanity.com/paper/1906.00232/) (2019)
- [Spectral Methods for Data Science: A Statistical Perspective](https://mlsanity.com/paper/2012.08496/) (2020)
- [Neural Networks Fail to Learn Periodic Functions and How to Fix It](https://mlsanity.com/paper/2006.08195/) (2020)
- [Minimax Estimation of Conditional Moment Models](https://mlsanity.com/paper/2006.07201/) (2020)
- [Learning Deep Features in Instrumental Variable Regression](https://mlsanity.com/paper/2010.07154/) (2020)
- [Deep Proxy Causal Learning and its Application to Confounded Bandit Policy Evaluation](https://mlsanity.com/paper/2106.03907/) (2021)
- [Sharp Spectral Rates for Koopman Operator Learning](https://mlsanity.com/paper/2302.02004/) (2023)
- [Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach](https://mlsanity.com/paper/2007.01290/) (2020)
- [Optimal Sup-norm Rates and Uniform Inference on Nonlinear Functionals of Nonparametric IV Regression](https://mlsanity.com/paper/1508.03365/) (2015)
- [NeuralEF: Deconstructing Kernels by Deep Neural Networks](https://mlsanity.com/paper/2205.00165/) (2022)
- [Neural Methods for Point-wise Dependency Estimation](https://mlsanity.com/paper/2006.05553/) (2020)
- [Functional Bilevel Optimization for Machine Learning](https://mlsanity.com/paper/2403.20233/) (2024)
- [An Information Theoretic Interpretation to Deep Neural Networks](https://mlsanity.com/paper/1905.06600/) (2019)
- [Instrumental Variable Regression via Kernel Maximum Moment Loss](https://mlsanity.com/paper/2010.07684/) (2020)
- [Neural Conditional Probability for Uncertainty Quantification](https://mlsanity.com/paper/2407.01171/) (2024)
- [Minimax Instrumental Variable Regression and L2 Convergence Guarantees without Identification or Closedness](https://mlsanity.com/paper/2302.05404/) (2023)
- [Spectral Representation Learning for Conditional Moment Models](https://mlsanity.com/paper/2210.16525/) (2022)
- [Spectral Representation for Causal Estimation with Hidden Confounders](https://mlsanity.com/paper/2407.10448/) (2024)
- [Nonparametric Instrumental Regression via Kernel Methods is Minimax Optimal](https://mlsanity.com/paper/2411.19653/) (2024)
- [Regularized DeepIV with Model Selection](https://mlsanity.com/paper/2403.04236/) (2024)
- [Fast Instrument Learning with Faster Rates](https://mlsanity.com/paper/2205.10772/) (2022)
- [Nonparametric Instrumental Variable Regression through Stochastic Approximate Gradients](https://mlsanity.com/paper/2402.05639/) (2024)
- [Correspondence Analysis Using Neural Networks](https://mlsanity.com/paper/1902.07828/) (2019)
- [Learning Decision Policies with Instrumental Variables through Double Machine Learning](https://mlsanity.com/paper/2405.08498/) (2024)
- [Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression](https://mlsanity.com/paper/2501.04898/) (2025)

## Built on by (most cited first, 4 of 4)

- [Outcome-Aware Spectral Feature Learning for Instrumental Variable Regression](https://mlsanity.com/paper/2512.00919/) (2025)
- [Toward Scalable and Valid Conditional Independence Testing with Spectral Representations](https://mlsanity.com/paper/2512.19510/) (2025)
- [Canonical correlation regression with noisy data](https://mlsanity.com/paper/2512.22697/) (2025)
- [Perturbative methods for non-parametric instrumental variable](https://mlsanity.com/paper/2606.00322/) (2026)

## Abstract (arXiv)

We address the problem of causal effect estimation in the presence of hidden confounders, using nonparametric instrumental variable (IV) regression. A leading strategy employs spectral features - that is, learned features spanning the top eigensubspaces of the operator linking treatments to instruments. We derive a generalization error bound for a two-stage least squares estimator based on spectral features, and gain insights into the method's performance and failure modes. We show that performance depends on two key factors, leading to a clear taxonomy of outcomes. In a good scenario, the approach is optimal. This occurs with strong spectral alignment, meaning the structural function is well-represented by the top eigenfunctions of the conditional operator, coupled with this operator's slow eigenvalue decay, indicating a strong instrument. Performance degrades in a bad scenario: spectral alignment remains strong, but rapid eigenvalue decay (indicating a weaker instrument) demands significantly more samples for effective feature learning. Finally, in the ugly scenario, weak spectral alignment causes the method to fail, regardless of the eigenvalues' characteristics. Our synthetic experiments empirically validate this taxonomy. We further introduce a practical procedure to estimate these spectral properties from data, allowing practitioners to diagnose which regime a given problem falls into. We apply this method to the dSprites dataset, demonstrating its utility.