# Scalable Subsampling Inference for Deep Neural Networks

arXiv: https://arxiv.org/abs/2405.08276 · HTML: https://mlsanity.com/paper/2405.08276/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 14 May 2024.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [Optimal approximation of continuous functions by very deep ReLU networks](https://mlsanity.com/paper/1802.03620/) (2018)
- [Rethinking Bias-Variance Trade-off for Generalization of Neural Networks](https://mlsanity.com/paper/2002.11328/) (2020)
- [The phase diagram of approximation rates for deep neural networks](https://mlsanity.com/paper/1906.09477/) (2019)
- [Estimation of a regression function on a manifold by fully connected deep neural networks](https://mlsanity.com/paper/2107.09532/) (2021)
- [Bootstrap Prediction Inference of Non-linear Autoregressive Models](https://mlsanity.com/paper/2306.04126/) (2023)
- [Model-free Bootstrap and Conformal Prediction in Regression: Conditionality, Conjecture Testing, and Pertinent Prediction Intervals](https://mlsanity.com/paper/2109.12156/) (2021)
- [On statistics, computation and scalability](https://mlsanity.com/paper/1309.7804/) (2013)
- [On the Subbagging Estimation for Massive Data](https://mlsanity.com/paper/2103.00631/) (2021)
- [Simple, Optimal Algorithms for Random Sampling Without Replacement](https://mlsanity.com/paper/2104.05091/) (2021)

## Built on by (most cited first, 5 of 5)

- [Deep Limit Model-free Prediction in Regression](https://mlsanity.com/paper/2408.09532/) (2024)
- [Fusion Sampling Validation in Data Partitioning for Machine Learning](https://mlsanity.com/paper/2508.01325/) (2025)
- [Calibration Prediction Interval for Non-parametric Regression and Neural Networks](https://mlsanity.com/paper/2509.02735/) (2025)
- [ProbRes: Volatility Learning for Probabilistic Time-Series Forecasting](https://mlsanity.com/paper/2606.02117/) (2026)
- [Prediction Inference of Time Series with Standard ReLU Deep Neural Networks](https://mlsanity.com/paper/2608.15362/) (2026)

## Abstract (arXiv)

Deep neural networks (DNN) has received increasing attention in machine learning applications in the last several years. Recently, a non-asymptotic error bound has been developed to measure the performance of the fully connected DNN estimator with ReLU activation functions for estimating regression models. The paper at hand gives a small improvement on the current error bound based on the latest results on the approximation ability of DNN. More importantly, however, a non-random subsampling technique--scalable subsampling--is applied to construct a `subagged' DNN estimator. Under regularity conditions, it is shown that the subagged DNN estimator is computationally efficient without sacrificing accuracy for either estimation or prediction tasks. Beyond point estimation/prediction, we propose different approaches to build confidence and prediction intervals based on the subagged DNN estimator. In addition to being asymptotically valid, the proposed confidence/prediction intervals appear to work well in finite samples. All in all, the scalable subsampling DNN estimator offers the complete package in terms of statistical inference, i.e., (a) computational efficiency; (b) point estimation/prediction accuracy; and (c) allowing for the construction of practically useful confidence and prediction intervals.