# Minimax Estimation of Conditional Moment Models

arXiv: https://arxiv.org/abs/2006.07201 · HTML: https://mlsanity.com/paper/2006.07201/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 30 of 83)

- [Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiency](https://mlsanity.com/paper/2102.02981/) (2021)
- [Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction](https://mlsanity.com/paper/2105.04544/) (2021)
- [Causal Inference Under Unmeasured Confounding With Negative Controls: A Minimax Learning Approach](https://mlsanity.com/paper/2103.14029/) (2021)
- [Proximal Reinforcement Learning: Efficient Off-Policy Evaluation in Partially Observed Markov Decision Processes](https://mlsanity.com/paper/2110.15332/) (2021)
- [A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2111.06784/) (2021)
- [Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning](https://mlsanity.com/paper/2102.09907/) (2021)
- [Provably Efficient Reinforcement Learning in Partially Observable Dynamical Systems](https://mlsanity.com/paper/2206.12020/) (2022)
- [Stochastic Extragradient: General Analysis and Improved Rates](https://mlsanity.com/paper/2111.08611/) (2021)
- [Causal Imitation Learning under Temporally Correlated Noise](https://mlsanity.com/paper/2202.01312/) (2022)
- [Long-term Causal Inference Under Persistent Confounding via Data Combination](https://mlsanity.com/paper/2202.07234/) (2022)
- [Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2205.13589/) (2022)
- [On Instrumental Variable Regression for Deep Offline Policy Evaluation](https://mlsanity.com/paper/2105.10148/) (2021)
- [Proximal Learning for Individualized Treatment Regimes Under Unmeasured Confounding](https://mlsanity.com/paper/2105.01187/) (2021)
- [Future-Dependent Value-Based Off-Policy Evaluation in POMDPs](https://mlsanity.com/paper/2207.13081/) (2022)
- [Minimax Kernel Machine Learning for a Class of Doubly Robust Functionals with Application to Proximal Causal Inference](https://mlsanity.com/paper/2104.02929/) (2021)
- [DoWhy: Addressing Challenges in Expressing and Validating Causal Assumptions](https://mlsanity.com/paper/2108.13518/) (2021)
- [Auto IV: Counterfactual Prediction via Automatic Instrumental Variable Decomposition](https://mlsanity.com/paper/2107.05884/) (2021)
- [Robust and Agnostic Learning of Conditional Distributional Treatment Effects](https://mlsanity.com/paper/2205.11486/) (2022)
- [Off-Policy Evaluation for Episodic Partially Observable Markov Decision Processes under Non-Parametric Models](https://mlsanity.com/paper/2209.10064/) (2022)
- [Demystifying Causal Features on Adversarial Examples and Causal Inoculation for Robust Network by Adversarial Instrumental Variable Regression](https://mlsanity.com/paper/2303.01052/) (2023)
- [Deep Learning Methods for Proximal Inference via Maximum Moment Restriction](https://mlsanity.com/paper/2205.09824/) (2022)
- [Minimax Optimal Kernel Operator Learning via Multilevel Training](https://mlsanity.com/paper/2209.14430/) (2022)
- [Minimax Instrumental Variable Regression and L2 Convergence Guarantees without Identification or Closedness](https://mlsanity.com/paper/2302.05404/) (2023)
- [Computationally Efficient PAC RL in POMDPs with Latent Determinism and Conditional Embeddings](https://mlsanity.com/paper/2206.12081/) (2022)
- [Inference on Strongly Identified Functionals of Weakly Identified Functions](https://mlsanity.com/paper/2208.08291/) (2022)
- [Spectral Representation Learning for Conditional Moment Models](https://mlsanity.com/paper/2210.16525/) (2022)
- [Spectral Representation for Causal Estimation with Hidden Confounders](https://mlsanity.com/paper/2407.10448/) (2024)
- [Instrumental Variables in Causal Inference and Machine Learning: A Survey](https://mlsanity.com/paper/2212.05778/) (2022)
- [Falsification of Internal and External Validity in Observational Studies via Conditional Moment Restrictions](https://mlsanity.com/paper/2301.13133/) (2023)
- [A Simple and General Debiased Machine Learning Theorem with Finite Sample Guarantees](https://mlsanity.com/paper/2105.15197/) (2021)

## Abstract (arXiv)

We develop an approach for estimating models described via conditional moment restrictions, with a prototypical application being non-parametric instrumental variable regression. We introduce a min-max criterion function, under which the estimation problem can be thought of as solving a zero-sum game between a modeler who is optimizing over the hypothesis space of the target model and an adversary who identifies violating moments over a test function space. We analyze the statistical estimation rate of the resulting estimator for arbitrary hypothesis spaces, with respect to an appropriate analogue of the mean squared error metric, for ill-posed inverse problems. We show that when the minimax criterion is regularized with a second moment penalty on the test function and the test function space is sufficiently rich, then the estimation rate scales with the critical radius of the hypothesis and test function spaces, a quantity which typically gives tight fast rates. Our main result follows from a novel localized Rademacher analysis of statistical learning problems defined via minimax objectives. We provide applications of our main results for several hypothesis spaces used in practice such as: reproducing kernel Hilbert spaces, high dimensional sparse linear functions, spaces defined via shape constraints, ensemble estimators such as random forests, and neural networks. For each of these applications we provide computationally efficient optimization methods for solving the corresponding minimax problem (e.g. stochastic first-order heuristics for neural networks). In several applications, we show how our modified mean squared error rate, combined with conditions that bound the ill-posedness of the inverse problem, lead to mean squared error rates. We conclude with an extensive experimental analysis of the proposed methods.