mlsanityData to 7 Oct 2026

Adaptive deep learning for nonlinear time series models

arXiv 2207.02546 ↗First posted 6 Jul 2022math.ST stat.ML stat.THcited by 11

Not on the map. Why?

Abstract (arXiv)

In this paper, we develop a general theory for adaptive nonparametric estimation of the mean function of a non-stationary and nonlinear time series model using deep neural networks (DNNs). We first consider two types of DNN estimators, non-penalized and sparse-penalized DNN estimators, and establish their generalization error bounds for general non-stationary time series. We then derive minimax lower bounds for estimating mean functions belonging to a wide class of nonlinear autoregressive (AR) models that include nonlinear generalized additive AR, single index, and threshold AR models. Building upon the results, we show that the sparse-penalized DNN estimator is adaptive and attains the minimax optimal rates up to a poly-logarithmic factor for many nonlinear AR models. Through numerical simulations, we demonstrate the usefulness of the DNN methods for estimating nonlinear AR models with intrinsic low-dimensional structures and discontinuous or rough mean functions, which is consistent with our theory.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 13 of its 69 references that are in the map. First posted on arXiv 6 Jul 2022.

Builds on

13 of 69 references.

References · 13

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  3. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  4. 2019Deep Neural Network Approximation Theorycited by 92
  5. 2019Deep ReLU network approximation of functions on a manifoldcited by 73
  6. 2018Deep Neural Networks Learn Non-Smooth Functions Effectivelycited by 57
  7. 2019Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov spacecited by 46
  8. 2017To tune or not to tune the number of trees in random forest?cited by 33
  9. 2019On the minimax optimality and superiority of deep neural network learning over sparse parameter spacescited by 24
  10. 2020Nonconvex sparse regularization for deep neural networks and its optimalitycited by 21
  11. 2020On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent datacited by 14
  12. 2020Estimation error analysis of deep learning on the regression problem on the variable exponent Besov spacecited by 13
1 more
  1. 2020Forecasting time series with encoder-decoder neural networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 1; 2026: 4.

11 papers.

  1. 2024Robust deep learning from weakly dependent datacited by 5
  2. 2024Minimax optimality of deep neural networks on dependent data via PAC-Bayes boundscited by 4
  3. 2024Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimalitycited by 3
  4. 2023Deep learning for psi-weakly dependent processescited by 2
  5. 2023Penalized deep neural networks estimator with general loss functions under weak dependencecited by 2
  6. 2024Trade-off Between Dependence and Complexity for Nonparametric Learning -- an Empirical Process Approachcited by 2
  7. 2025A general framework for deep learningcited by 2
  8. 2026Deep regression learning from dependent observations with minimum error entropy principlecited by 0
  9. 2026Rate-optimal neural boundary detection from unlabeled noisy imagescited by 0
  10. 2026Adaptive deep nonparametric regression from dependent data under covariate shiftcited by 0
  11. 2026Prediction Inference of Time Series with Standard ReLU Deep Neural Networkscited by 0