# Deep Neural Networks for Estimation and Inference

arXiv: https://arxiv.org/abs/1809.09953 · HTML: https://mlsanity.com/paper/1809.09953/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2018.

## Built on by (most cited first, 30 of 62)

- [VCNet and Functional Targeted Regularization For Learning Causal Effects of Continuous Treatments](https://mlsanity.com/paper/2103.07861/) (2021)
- [Criteria for Classifying Forecasting Methods](https://mlsanity.com/paper/2212.03523/) (2022)
- [Deep Nonparametric Estimation of Operators between Infinite Dimensional Spaces](https://mlsanity.com/paper/2201.00217/) (2022)
- [RieszNet and ForestRiesz: Automatic Debiased Machine Learning with Neural Nets and Random Forests](https://mlsanity.com/paper/2110.03031/) (2021)
- [Uncertainty quantification in the Bradley-Terry-Luce model](https://mlsanity.com/paper/2110.03874/) (2021)
- [How do noise tails impact on deep ReLU networks?](https://mlsanity.com/paper/2203.10418/) (2022)
- [Deep Operator Learning Lessens the Curse of Dimensionality for PDEs](https://mlsanity.com/paper/2301.12227/) (2023)
- [Inference on Strongly Identified Functionals of Weakly Identified Functions](https://mlsanity.com/paper/2208.08291/) (2022)
- [DoubleMLDeep: Estimation of Causal Effects with Multimodal Data](https://mlsanity.com/paper/2402.01785/) (2024)
- [Generating Synthetic Text Data to Evaluate Causal Inference Methods](https://mlsanity.com/paper/2102.05638/) (2021)
- [Testing for the Markov Property in Time Series via Deep Conditional Generative Learning](https://mlsanity.com/paper/2305.19244/) (2023)
- [Sample Complexity of Offline Reinforcement Learning with Deep ReLU Networks](https://mlsanity.com/paper/2103.06671/) (2021)
- [RCT Rejection Sampling for Causal Estimation Evaluation](https://mlsanity.com/paper/2307.15176/) (2023)
- [Automatic Debiased Machine Learning for Smooth Functionals of Nonparametric M-Estimands](https://mlsanity.com/paper/2501.11868/) (2025)
- [Calibrating multi-dimensional complex ODE from noisy data via deep neural networks](https://mlsanity.com/paper/2106.03591/) (2021)
- [Deep Feature Screening: Feature Selection for Ultra High-Dimensional Data via Deep Neural Networks](https://mlsanity.com/paper/2204.01682/) (2022)
- [Sup-Norm Convergence of Deep Neural Network Estimator for Nonparametric Regression by Adversarial Training](https://mlsanity.com/paper/2307.04042/) (2023)
- [Estimation of a regression function on a manifold by fully connected deep neural networks](https://mlsanity.com/paper/2107.09532/) (2021)
- [Covariate-Balancing-Aware Interpretable Deep Learning models for Treatment Effect Estimation](https://mlsanity.com/paper/2203.03185/) (2022)
- [On the Pointwise Behavior of Recursive Partitioning and Its Implications for Heterogeneous Causal Effect Estimation](https://mlsanity.com/paper/2211.10805/) (2022)
- [Improving the Finite Sample Estimation of Average Treatment Effects using Double/Debiased Machine Learning with Propensity Score Calibration](https://mlsanity.com/paper/2409.04874/) (2024)
- [Unlabeled Data Help: Minimax Analysis and Adversarial Robustness](https://mlsanity.com/paper/2202.06996/) (2022)
- [Deep Gaussian Mixture Ensembles](https://mlsanity.com/paper/2306.07235/) (2023)
- [Distributionally Robust Learning for Multi-source Unsupervised Domain Adaptation](https://mlsanity.com/paper/2309.02211/) (2023)
- [Off-policy Evaluation with Deeply-abstracted States](https://mlsanity.com/paper/2406.19531/) (2024)
- [Estimation and Applications of Quantiles in Deep Binary Classification](https://mlsanity.com/paper/2102.06575/) (2021)
- [Individual Treatment Effect Estimation Through Controlled Neural Network Training in Two Stages](https://mlsanity.com/paper/2201.08559/) (2022)
- [Estimation of Non-Crossing Quantile Regression Process with Deep ReQU Neural Networks](https://mlsanity.com/paper/2207.10442/) (2022)
- [Bayesian Neural Networks for Macroeconomic Analysis](https://mlsanity.com/paper/2211.04752/) (2022)
- [Nonasymptotic theory for two-layer neural networks: Beyond the bias-variance trade-off](https://mlsanity.com/paper/2106.04795/) (2021)

## Abstract (arXiv)

We study deep neural networks and their use in semiparametric inference. We establish novel rates of convergence for deep feedforward neural nets. Our new rates are sufficiently fast (in some cases minimax optimal) to allow us to establish valid second-step inference after first-step estimation with deep learning, a result also new to the literature. Our estimation rates and semiparametric inference results handle the current standard architecture: fully connected feedforward neural networks (multi-layer perceptrons), with the now-common rectified linear unit activation function and a depth explicitly diverging with the sample size. We discuss other architectures as well, including fixed-width, very deep networks. We establish nonasymptotic bounds for these deep nets for a general class of nonparametric regression-type loss functions, which includes as special cases least squares, logistic regression, and other generalized linear models. We then apply our theory to develop semiparametric inference, focusing on causal parameters for concreteness, such as treatment effects, expected welfare, and decomposition effects. Inference in many other semiparametric contexts can be readily obtained. We demonstrate the effectiveness of deep learning with a Monte Carlo analysis and an empirical application to direct mail marketing.