# Robust deep learning from weakly dependent data

arXiv: https://arxiv.org/abs/2405.05081 · HTML: https://mlsanity.com/paper/2405.05081/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 8 May 2024.

## Builds on (most cited first, 20 of 20)

- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Deep ReLU network approximation of functions on a manifold](https://mlsanity.com/paper/1908.00695/) (2019)
- [Smooth function approximation by deep neural networks with general activation functions](https://mlsanity.com/paper/1906.06903/) (2019)
- [How do noise tails impact on deep ReLU networks?](https://mlsanity.com/paper/2203.10418/) (2022)
- [Nonconvex sparse regularization for deep neural networks and its optimality](https://mlsanity.com/paper/2003.11769/) (2020)
- [Advantage of Deep Neural Networks for Estimating Functions with Singularity on Hypersurfaces](https://mlsanity.com/paper/2011.02256/) (2020)
- [Theoretical analysis of deep neural networks for temporally dependent observations](https://mlsanity.com/paper/2210.11530/) (2022)
- [Classification with Deep Neural Networks and Logistic Loss](https://mlsanity.com/paper/2307.16792/) (2023)
- [On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent data](https://mlsanity.com/paper/2011.00328/) (2020)
- [Estimation error analysis of deep learning on the regression problem on the variable exponent Besov space](https://mlsanity.com/paper/2009.11285/) (2020)
- [Adaptive deep learning for nonlinear time series models](https://mlsanity.com/paper/2207.02546/) (2022)
- [Risk Bounds for Robust Deep Learning](https://mlsanity.com/paper/2009.06202/) (2020)
- [Excess risk bound for deep learning under weak dependence](https://mlsanity.com/paper/2302.07503/) (2023)
- [Sparse-penalized deep neural networks estimator under weak dependence](https://mlsanity.com/paper/2303.01406/) (2023)
- [Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption](https://mlsanity.com/paper/2201.03182/) (2022)
- [Learning with Correntropy-induced Losses for Regression with Mixture of Symmetric Stable Noise](https://mlsanity.com/paper/1803.00183/) (2018)
- [Robust Deep Neural Network Estimation for Multi-dimensional Functional Data](https://mlsanity.com/paper/2205.09604/) (2022)
- [Robust Sensible Adversarial Learning of Deep Neural Networks for Image Classification](https://mlsanity.com/paper/2205.10457/) (2022)
- [Penalized deep neural networks estimator with general loss functions under weak dependence](https://mlsanity.com/paper/2305.06230/) (2023)
- [Inheritance of strong mixing and weak dependence under renewal sampling](https://mlsanity.com/paper/2007.00574/) (2020)

## Built on by (most cited first, 5 of 5)

- [Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimality](https://mlsanity.com/paper/2406.08321/) (2024)
- [A general framework for deep learning](https://mlsanity.com/paper/2512.23425/) (2025)
- [RCDM: Enabling Robustness for Conditional Diffusion Model](https://mlsanity.com/paper/2408.02710/) (2024)
- [Deep regression learning from dependent observations with minimum error entropy principle](https://mlsanity.com/paper/2603.11138/) (2026)
- [Adaptive deep nonparametric regression from dependent data under covariate shift](https://mlsanity.com/paper/2607.20309/) (2026)

## Abstract (arXiv)

Recent developments on deep learning established some theoretical properties of deep neural networks estimators. However, most of the existing works on this topic are restricted to bounded loss functions or (sub)-Gaussian or bounded input. This paper considers robust deep learning from weakly dependent observations, with unbounded loss function and unbounded input/output. It is only assumed that the output variable has a finite r order moment, with r >1. Non asymptotic bounds for the expected excess risk of the deep neural network estimator are established under strong mixing, and psi-weak dependence assumptions on the observations. We derive a relationship between these bounds and r, and when the data have moments of any order (that is r=∞), the convergence rate is close to some well-known results. When the target predictor belongs to the class of H"older smooth functions with sufficiently large smoothness index, the rate of the expected excess risk for exponentially strongly mixing data is close to or as same as those for obtained with i.i.d. samples. Application to robust nonparametric regression and robust nonparametric autoregression are considered. The simulation study for models with heavy-tailed errors shows that, robust estimators with absolute loss and Huber loss function outperform the least squares method.