mlsanityData to 14 Oct 2026

Spectral Representation Learning for Conditional Moment Models

arXiv 2210.16525 ↗First posted 29 Oct 2022stat.ML cs.LG econ.EMcited by 12

Not on the map. Why?

Abstract (arXiv)

Many problems in causal inference and economics can be formulated in the framework of conditional moment models, which characterize the target function through a collection of conditional moment restrictions. For nonparametric conditional moment models, efficient estimation often relies on preimposed conditions on various measures of ill-posedness of the hypothesis space, which are hard to validate when flexible models are used. In this work, we address this issue by proposing a procedure that automatically learns representations with controlled measures of ill-posedness. Our method approximates a linear representation defined by the spectral decomposition of a conditional expectation operator, which can be used for kernelized estimators and is known to facilitate minimax optimal estimation in certain settings. We show this representation can be efficiently estimated from data, and establish L2 consistency for the resulting estimator. We evaluate the proposed method on proximal causal inference tasks, exhibiting promising performance on high-dimensional, semi-synthetic data.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 41 of its 86 references that are in the map. First posted on arXiv 29 Oct 2022.

Builds on

41 of 86 references.

References · 41

  1. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  2. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
  3. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 313
  4. 2018Gaussian Processes and Kernel Methods: A Review on Connections and Equivalencescited by 192
  5. 2020Predicting What You Already Know Helps: Provable Self-Supervised Learningcited by 152
  6. 2019Orthogonal Statistical Learningcited by 151
  7. 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 149
  8. 2020Contrastive learning, multi-view redundancy, and linear modelscited by 146
  9. 2016Identifying Causal Effects With Proxy Variables of an Unmeasured Confoundercited by 139
  10. 2019Kernel Instrumental Variable Regressioncited by 118
  11. 2020An Introduction to Proximal Causal Learningcited by 95
  12. 2019Deep Generalized Method of Moments for Instrumental Variable Analysiscited by 88
29 more
  1. 2020A Measure-Theoretic Approach to Kernel Conditional Mean Embeddingscited by 86
  2. 2020Minimax Estimation of Conditional Moment Modelscited by 83
  3. 2020Mat'ern Gaussian processes on Riemannian manifoldscited by 82
  4. 2022Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spacescited by 67
  5. 2020Learning Deep Features in Instrumental Variable Regressioncited by 66
  6. 2021Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restrictioncited by 55
  7. 2022Making Linear MDPs Practical via Contrastive Representation Learningcited by 52
  8. 2021Causal Inference Under Unmeasured Confounding With Negative Controls: A Minimax Learning Approachcited by 44
  9. 2021Deep Proxy Causal Learning and its Application to Confounded Bandit Policy Evaluationcited by 40
  10. 2020EigenGame: PCA as a Nash Equilibriumcited by 39
  11. 2021A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processescited by 36
  12. 2018Spectral Inference Networks: Unifying Deep and Spectral Learningcited by 35
  13. 2018Adversarial Generalized Method of Momentscited by 34
  14. 2022Contrastive Learning Can Find An Optimal Basis For Approximately View-Invariant Functionscited by 32
  15. 2020Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approachcited by 31
  16. 2015Optimal Sup-norm Rates and Uniform Inference on Nonlinear Functionals of Nonparametric IV Regressioncited by 30
  17. 2022Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learningcited by 26
  18. 2021On Instrumental Variable Regression for Deep Offline Policy Evaluationcited by 22
  19. 2020Kernel Conditional Moment Test via Maximum Moment Restrictioncited by 18
  20. 2022Neural Eigenfunctions Are Structured Representation Learnerscited by 16
  21. 2019Reproducing kernel Hilbert spaces on manifolds: Sobolev and Diffusion spacescited by 14
  22. 2019What can be estimated? Identifiability, estimability, causal inference and ill-posed inverse problemscited by 11
  23. 2013Regularization Methods for High-Dimensional Instrumental Variables Regression With an Application to Genetical Genomicscited by 9
  24. 2021Asymptotic Theory for IV-Based Reinforcement Learning with Potential Endogeneitycited by 8
  25. 2022Functional Generalized Empirical Likelihood Estimation for Conditional Moment Restrictionscited by 8
  26. 2021Sobolev Norm Learning Rates for Conditional Mean Embeddingscited by 7
  27. 2021Learning Causal Models from Conditional Moment Restrictions by Importance Weightingcited by 7
  28. 2022Fast Instrument Learning with Faster Ratescited by 6
  29. 2021Quasi-Bayesian Dual Instrumental Variable Regressioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 5; 2026: 2.

12 papers.

  1. 2023Sharp Spectral Rates for Koopman Operator Learningcited by 36
  2. 2023Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regressioncited by 20
  3. 2024Neural Conditional Probability for Uncertainty Quantificationcited by 14
  4. 2024Spectral Representation for Causal Estimation with Hidden Confounderscited by 12
  5. 2025Self-Supervised Evolution Operator Learning for High-Dimensional Dynamical Systemscited by 7
  6. 2023Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysiscited by 6
  7. 2025Efficient Parametric SVD of Koopman Operator for Stochastic Dynamical Systemscited by 5
  8. 2025Demystifying Spectral Feature Learning for Instrumental Variable Regressioncited by 4
  9. 2025Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regressioncited by 3
  10. 2025Toward Scalable and Valid Conditional Independence Testing with Spectral Representationscited by 1
  11. 2026Spectral Ghost in Representation Learning: from Component Analysis to Self-Supervised Learningcited by 0
  12. 2026Learning Conditional Expectation Operators via Functional Newton Updatescited by 0