mlsanityData to 14 Oct 2026

Robust deep learning from weakly dependent data

arXiv 2405.05081 ↗First posted 8 May 2024stat.ML cs.LG math.STcited by 5

Not on the map. Why?

Abstract (arXiv)

Recent developments on deep learning established some theoretical properties of deep neural networks estimators. However, most of the existing works on this topic are restricted to bounded loss functions or (sub)-Gaussian or bounded input. This paper considers robust deep learning from weakly dependent observations, with unbounded loss function and unbounded input/output. It is only assumed that the output variable has a finite r order moment, with r >1. Non asymptotic bounds for the expected excess risk of the deep neural network estimator are established under strong mixing, and psi-weak dependence assumptions on the observations. We derive a relationship between these bounds and r, and when the data have moments of any order (that is r=∞), the convergence rate is close to some well-known results. When the target predictor belongs to the class of H"older smooth functions with sufficiently large smoothness index, the rate of the expected excess risk for exponentially strongly mixing data is close to or as same as those for obtained with i.i.d. samples. Application to robust nonparametric regression and robust nonparametric autoregression are considered. The simulation study for models with heavy-tailed errors shows that, robust estimators with absolute loss and Huber loss function outperform the least squares method.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 20 of its 46 references that are in the map. First posted on arXiv 8 May 2024.

Builds on

20 of 46 references.

References · 20

  1. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
  2. 2019Deep ReLU network approximation of functions on a manifoldcited by 74
  3. 2019Smooth function approximation by deep neural networks with general activation functionscited by 30
  4. 2022How do noise tails impact on deep ReLU networks?cited by 25
  5. 2020Nonconvex sparse regularization for deep neural networks and its optimalitycited by 21
  6. 2020Advantage of Deep Neural Networks for Estimating Functions with Singularity on Hypersurfacescited by 17
  7. 2022Theoretical analysis of deep neural networks for temporally dependent observationscited by 16
  8. 2023Classification with Deep Neural Networks and Logistic Losscited by 16
  9. 2020On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent datacited by 14
  10. 2020Estimation error analysis of deep learning on the regression problem on the variable exponent Besov spacecited by 13
  11. 2022Adaptive deep learning for nonlinear time series modelscited by 11
  12. 2020Risk Bounds for Robust Deep Learningcited by 9
8 more
  1. 2023Excess risk bound for deep learning under weak dependencecited by 9
  2. 2023Sparse-penalized deep neural networks estimator under weak dependencecited by 8
  3. 2022Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumptioncited by 6
  4. 2018Learning with Correntropy-induced Losses for Regression with Mixture of Symmetric Stable Noisecited by 2
  5. 2022Robust Deep Neural Network Estimation for Multi-dimensional Functional Datacited by 2
  6. 2022Robust Sensible Adversarial Learning of Deep Neural Networks for Image Classificationcited by 2
  7. 2023Penalized deep neural networks estimator with general loss functions under weak dependencecited by 2
  8. 2020Inheritance of strong mixing and weak dependence under renewal samplingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 2.

5 papers.

  1. 2024Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimalitycited by 3
  2. 2025A general framework for deep learningcited by 2
  3. 2024RCDM: Enabling Robustness for Conditional Diffusion Modelcited by 0
  4. 2026Deep regression learning from dependent observations with minimum error entropy principlecited by 0
  5. 2026Adaptive deep nonparametric regression from dependent data under covariate shiftcited by 0