# Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimality

arXiv: https://arxiv.org/abs/2406.08321 · HTML: https://mlsanity.com/paper/2406.08321/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 12 Jun 2024.

## Builds on (most cited first, 19 of 19)

- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Deep Neural Networks Learn Non-Smooth Functions Effectively](https://mlsanity.com/paper/1802.04474/) (2018)
- [Fast convergence rates of deep neural networks for classification](https://mlsanity.com/paper/1812.03599/) (2018)
- [Smooth function approximation by deep neural networks with general activation functions](https://mlsanity.com/paper/1906.06903/) (2019)
- [How do noise tails impact on deep ReLU networks?](https://mlsanity.com/paper/2203.10418/) (2022)
- [On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces](https://mlsanity.com/paper/1905.09195/) (2019)
- [Quantile regression with deep ReLU Networks: Estimators and minimax rates](https://mlsanity.com/paper/2010.08236/) (2020)
- [Nonconvex sparse regularization for deep neural networks and its optimality](https://mlsanity.com/paper/2003.11769/) (2020)
- [Advantage of Deep Neural Networks for Estimating Functions with Singularity on Hypersurfaces](https://mlsanity.com/paper/2011.02256/) (2020)
- [Theoretical analysis of deep neural networks for temporally dependent observations](https://mlsanity.com/paper/2210.11530/) (2022)
- [Classification with Deep Neural Networks and Logistic Loss](https://mlsanity.com/paper/2307.16792/) (2023)
- [On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent data](https://mlsanity.com/paper/2011.00328/) (2020)
- [Estimation error analysis of deep learning on the regression problem on the variable exponent Besov space](https://mlsanity.com/paper/2009.11285/) (2020)
- [Adaptive deep learning for nonlinear time series models](https://mlsanity.com/paper/2207.02546/) (2022)
- [Excess risk bound for deep learning under weak dependence](https://mlsanity.com/paper/2302.07503/) (2023)
- [Sparse-penalized deep neural networks estimator under weak dependence](https://mlsanity.com/paper/2303.01406/) (2023)
- [Learning theory estimates with observations from general stationary stochastic processes](https://mlsanity.com/paper/1605.02887/) (2016)
- [Robust deep learning from weakly dependent data](https://mlsanity.com/paper/2405.05081/) (2024)
- [A novel nonconvex, smooth-at-origin penalty for statistical learning](https://mlsanity.com/paper/2204.03123/) (2022)

## Built on by (most cited first, 3 of 3)

- [A general framework for deep learning](https://mlsanity.com/paper/2512.23425/) (2025)
- [Deep regression learning from dependent observations with minimum error entropy principle](https://mlsanity.com/paper/2603.11138/) (2026)
- [Adaptive deep nonparametric regression from dependent data under covariate shift](https://mlsanity.com/paper/2607.20309/) (2026)

## Abstract (arXiv)

The explicit regularization and optimality of deep neural networks estimators from independent data have made considerable progress recently. The study of such properties on dependent data is still a challenge. In this paper, we carry out deep learning from strongly mixing observations, and deal with the squared and a broad class of loss functions. We consider sparse-penalized regularization for deep neural network predictor. For a general framework that includes, regression estimation, classification, time series prediction,cdots, oracle inequality for the expected excess risk is established and a bound on the class of H"older smooth functions is provided. For nonparametric regression from strong mixing data and sub-exponentially error, we provide an oracle inequality for the L2 error and investigate an upper bound of this error on a class of H"older composition functions. For the specific case of nonparametric autoregression with Gaussian and Laplace errors, a lower bound of the L2 error on this H"older composition class is established. Up to logarithmic factor, this bound matches its upper bound; so, the deep neural network estimator attains the minimax optimal rate.