Statistical Properties of Deep Neural Networks with Dependent Data

This paper develops theory for deep neural network (DNN) estimators under dependent data. To provide theory applicable to a variety of DNN-based estimators, I first establish nonasymptotic probability bounds on the theoretical and empirical L2-errors of nonparametric sieve estimators for a general class of estimation problems under possibly nonstationary β-mixing data taking values in unbounded sets. I then apply the theory to fully connected and convolutional DNN estimators without bounds or sparsity restrictions on the DNN weights. For both DNN classes, I derive general results when the function to be estimated is H"older smooth and the data are nonstationary, subgaussian, and β-mixing with either exponential or polynomial decay. I then specialize these to nonparametric regression, logistic regression, and quantile regression settings. Under exponential β-mixing, the resulting estimators attain the nonparametric minimax rate of Stone (1982) up to logarithmic factors.