mlsanityData to 7 Oct 2026

Theoretical analysis of deep neural networks for temporally dependent observations

arXiv 2210.11530 ↗First posted 20 Oct 2022stat.ML cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Deep neural networks are powerful tools to model observations over time with non-linear patterns. Despite the widespread use of neural networks in such settings, most theoretical developments of deep neural networks are under the assumption of independent observations, and theoretical results for temporally dependent observations are scarce. To bridge this gap, we study theoretical properties of deep neural networks on modeling non-linear time series data. Specifically, non-asymptotic bounds for prediction error of (sparse) feed-forward neural network with ReLU activation function is established under mixing-type assumptions. These assumptions are mild such that they include a wide range of time series models including auto-regressive models. Compared to independent observations, established convergence rates have additional logarithmic factors to compensate for additional complexity due to dependence among data points. The theoretical results are supported via various numerical simulation settings as well as an application to a macroeconomic data set.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 10 of its 32 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

10 of 32 references.

References · 10

  1. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  2. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  3. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  4. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  5. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  6. 2014The Loss Surfaces of Multilayer Networkscited by 278
  7. 2019A Selective Overview of Deep Learningcited by 31
  8. 2018Approximation and Estimation for High-Dimensional Deep Learning Networkscited by 22
  9. 2021Generalization Error Bounds on Deep Learning with Markov Datasetscited by 9
  10. 2020Modeling of time series using random forests: theoretical developmentscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 5; 2025: 4; 2026: 3.

16 papers.

Sort:

  1. 2023Excess risk bound for deep learning under weak dependencecited by 9
  2. 2023Sparse-penalized deep neural networks estimator under weak dependencecited by 8
  3. 2024Robust deep learning from weakly dependent datacited by 5
  4. 2024Minimax optimality of deep neural networks on dependent data via PAC-Bayes boundscited by 4
  5. 2025Double Machine Learning for Causal Inference under Shared-State Interferencecited by 4
  6. 2024Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimalitycited by 3
  7. 2023Deep learning for psi-weakly dependent processescited by 2
  8. 2023Penalized deep neural networks estimator with general loss functions under weak dependencecited by 2
  9. 2024Trade-off Between Dependence and Complexity for Nonparametric Learning -- an Empirical Process Approachcited by 2
  10. 2024Semiparametric inference for impulse response functions using double/debiased machine learningcited by 1
  11. 2025Survey Transfer Learning: Recycling Data with Silicon Responsescited by 0
  12. 2025Neural Network-Based Change Point Detection for Large-Scale Time-Evolving Datacited by 0
4 more
  1. 2025Risk Bounds For Distributional Regressioncited by 0
  2. 2026Deep regression learning from dependent observations with minimum error entropy principlecited by 0
  3. 2026Beyond the Independence Assumption: Finite-Sample Guarantees for Deep Q-Learning under τ-Mixingcited by 0
  4. 2026Adaptive deep nonparametric regression from dependent data under covariate shiftcited by 0