Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimality
Not on the map. Why?
Abstract (arXiv)
The explicit regularization and optimality of deep neural networks estimators from independent data have made considerable progress recently. The study of such properties on dependent data is still a challenge. In this paper, we carry out deep learning from strongly mixing observations, and deal with the squared and a broad class of loss functions. We consider sparse-penalized regularization for deep neural network predictor. For a general framework that includes, regression estimation, classification, time series prediction,cdots, oracle inequality for the expected excess risk is established and a bound on the class of H"older smooth functions is provided. For nonparametric regression from strong mixing data and sub-exponentially error, we provide an oracle inequality for the L2 error and investigate an upper bound of this error on a class of H"older composition functions. For the specific case of nonparametric autoregression with Gaussian and Laplace errors, a lower bound of the L2 error on this H"older composition class is established. Up to logarithmic factor, this bound matches its upper bound; so, the deep neural network estimator attains the minimax optimal rate.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 19 of its 45 references that are in the map. First posted on arXiv 12 Jun 2024.
Builds on
19 of 45 references.
References · 19
- 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
- 2018Deep Neural Networks Learn Non-Smooth Functions Effectivelycited by 57
- 2018Fast convergence rates of deep neural networks for classificationcited by 46
- 2019Smooth function approximation by deep neural networks with general activation functionscited by 30
- 2022How do noise tails impact on deep ReLU networks?cited by 25
- 2019On the minimax optimality and superiority of deep neural network learning over sparse parameter spacescited by 24
- 2020Quantile regression with deep ReLU Networks: Estimators and minimax ratescited by 24
- 2020Nonconvex sparse regularization for deep neural networks and its optimalitycited by 21
- 2020Advantage of Deep Neural Networks for Estimating Functions with Singularity on Hypersurfacescited by 17
- 2022Theoretical analysis of deep neural networks for temporally dependent observationscited by 16
- 2023Classification with Deep Neural Networks and Logistic Losscited by 16
- 2020On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent datacited by 14
7 more
- 2020Estimation error analysis of deep learning on the regression problem on the variable exponent Besov spacecited by 13
- 2022Adaptive deep learning for nonlinear time series modelscited by 11
- 2023Excess risk bound for deep learning under weak dependencecited by 9
- 2023Sparse-penalized deep neural networks estimator under weak dependencecited by 8
- 2016Learning theory estimates with observations from general stationary stochastic processescited by 7
- 2024Robust deep learning from weakly dependent datacited by 5
- 2022A novel nonconvex, smooth-at-origin penalty for statistical learningcited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.