# Deep Neural Networks for Nonparametric Interaction Models with Diverging Dimension

arXiv: https://arxiv.org/abs/2302.05851 · HTML: https://mlsanity.com/paper/2302.05851/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 62 references that are in the map.
- First posted on arXiv 12 Feb 2023.

## Builds on (most cited first, 8 of 8)

- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [Deep Network Approximation for Smooth Functions](https://mlsanity.com/paper/2001.03040/) (2020)
- [Optimal Approximation Rate of ReLU Networks in terms of Width and Depth](https://mlsanity.com/paper/2103.00502/) (2021)
- [Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression](https://mlsanity.com/paper/2210.02002/) (2022)
- [How do noise tails impact on deep ReLU networks?](https://mlsanity.com/paper/2203.10418/) (2022)
- [Minimax Optimal Rates of Estimation in High Dimensional Additive Models: Universal Phase Transition](https://mlsanity.com/paper/1503.02817/) (2015)

## Built on by (most cited first, 15 of 15)

- [UTOPIA: Universally Trainable Optimal Prediction Intervals Aggregation](https://mlsanity.com/paper/2306.16549/) (2023)
- [Deep Transfer Learning: Model Framework and Error Analysis](https://mlsanity.com/paper/2410.09383/) (2024)
- [Trade-off Between Dependence and Complexity for Nonparametric Learning -- an Empirical Process Approach](https://mlsanity.com/paper/2401.08978/) (2024)
- [Model Free Prediction with Uncertainty Assessment](https://mlsanity.com/paper/2405.12684/) (2024)
- [Factor Informed Double Deep Learning For Average Treatment Effect Estimation](https://mlsanity.com/paper/2508.17136/) (2025)
- [Improving Mixup Calibration with Wasserstein Distributionally Robust Optimization](https://mlsanity.com/paper/2506.17874/) (2025)
- [CINDES: Classification induced neural density estimator and simulator](https://mlsanity.com/paper/2510.00367/) (2025)
- [Generalized Functional ANOVA: A Complete Theoretical Framework](https://mlsanity.com/paper/2605.18422/) (2026)
- [Minimax Signal Detection in Sparse Additive Models](https://mlsanity.com/paper/2304.09398/) (2023)
- [Statistically guided deep learning](https://mlsanity.com/paper/2504.08489/) (2025)
- [Boosting Statistic Learning with Synthetic Data from Pretrained Large Models](https://mlsanity.com/paper/2505.04992/) (2025)
- [Minimax Rates for Learning Pairwise Interactions in Attention-Style Models](https://mlsanity.com/paper/2510.11789/) (2025)
- [Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activations](https://mlsanity.com/paper/2606.05599/) (2026)
- [Minimax Theory of Likelihood-Based Deep Learning for Speckle Regression](https://mlsanity.com/paper/2607.14064/) (2026)
- [Minimax Additive Regression under Unknown Dependent Designs](https://mlsanity.com/paper/2609.39212/) (2026)

## Abstract (arXiv)

Deep neural networks have achieved tremendous success due to their representation power and adaptation to low-dimensional structures. Their potential for estimating structured regression functions has been recently established in the literature. However, most of the studies require the input dimension to be fixed and consequently ignore the effect of dimension on the rate of convergence and hamper their applications to modern big data with high dimensionality. In this paper, we bridge this gap by analyzing a kth order nonparametric interaction model in both growing dimension scenarios (d grows with n but at a slower rate) and in high dimension (d gtrsim n). In the latter case, sparsity assumptions and associated regularization are required in order to obtain optimal rates of convergence. A new challenge in diverging dimension setting is in calculation mean-square error, the covariance terms among estimated additive components are an order of magnitude larger than those of the variances and they can deteriorate statistical properties without proper care. We introduce a critical debiasing technique to amend the problem. We show that under certain standard assumptions, debiased deep neural networks achieve a minimax optimal rate both in terms of (n, d). Our proof techniques rely crucially on a novel debiasing technique that makes the covariances of additive components negligible in the mean-square error calculation. In addition, we establish the matching lower bounds.