# Theoretical analysis of deep neural networks for temporally dependent observations

arXiv: https://arxiv.org/abs/2210.11530 · HTML: https://mlsanity.com/paper/2210.11530/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 32 references that are in the map.
- First posted on arXiv 20 Oct 2022.

## Builds on (most cited first, 10 of 10)

- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Norm-Based Capacity Control in Neural Networks](https://mlsanity.com/paper/1503.00036/) (2015)
- [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017)
- [The Loss Surfaces of Multilayer Networks](https://mlsanity.com/paper/1412.0233/) (2014)
- [A Selective Overview of Deep Learning](https://mlsanity.com/paper/1904.05526/) (2019)
- [Approximation and Estimation for High-Dimensional Deep Learning Networks](https://mlsanity.com/paper/1809.03090/) (2018)
- [Generalization Error Bounds on Deep Learning with Markov Datasets](https://mlsanity.com/paper/2201.11059/) (2021)
- [Modeling of time series using random forests: theoretical developments](https://mlsanity.com/paper/2008.02479/) (2020)

## Built on by (most cited first, 16 of 16)

- [Excess risk bound for deep learning under weak dependence](https://mlsanity.com/paper/2302.07503/) (2023)
- [Sparse-penalized deep neural networks estimator under weak dependence](https://mlsanity.com/paper/2303.01406/) (2023)
- [Robust deep learning from weakly dependent data](https://mlsanity.com/paper/2405.05081/) (2024)
- [Minimax optimality of deep neural networks on dependent data via PAC-Bayes bounds](https://mlsanity.com/paper/2410.21702/) (2024)
- [Double Machine Learning for Causal Inference under Shared-State Interference](https://mlsanity.com/paper/2504.08836/) (2025)
- [Trade-off Between Dependence and Complexity for Nonparametric Learning -- an Empirical Process Approach](https://mlsanity.com/paper/2401.08978/) (2024)
- [Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimality](https://mlsanity.com/paper/2406.08321/) (2024)
- [Deep learning for psi-weakly dependent processes](https://mlsanity.com/paper/2302.00333/) (2023)
- [Penalized deep neural networks estimator with general loss functions under weak dependence](https://mlsanity.com/paper/2305.06230/) (2023)
- [Semiparametric inference for impulse response functions using double/debiased machine learning](https://mlsanity.com/paper/2411.10009/) (2024)
- [Survey Transfer Learning: Recycling Data with Silicon Responses](https://mlsanity.com/paper/2501.06577/) (2025)
- [Neural Network-Based Change Point Detection for Large-Scale Time-Evolving Data](https://mlsanity.com/paper/2503.09541/) (2025)
- [Risk Bounds For Distributional Regression](https://mlsanity.com/paper/2505.09075/) (2025)
- [Deep regression learning from dependent observations with minimum error entropy principle](https://mlsanity.com/paper/2603.11138/) (2026)
- [Beyond the Independence Assumption: Finite-Sample Guarantees for Deep Q-Learning under τ-Mixing](https://mlsanity.com/paper/2605.06373/) (2026)
- [Adaptive deep nonparametric regression from dependent data under covariate shift](https://mlsanity.com/paper/2607.20309/) (2026)

## Abstract (arXiv)

Deep neural networks are powerful tools to model observations over time with non-linear patterns. Despite the widespread use of neural networks in such settings, most theoretical developments of deep neural networks are under the assumption of independent observations, and theoretical results for temporally dependent observations are scarce. To bridge this gap, we study theoretical properties of deep neural networks on modeling non-linear time series data. Specifically, non-asymptotic bounds for prediction error of (sparse) feed-forward neural network with ReLU activation function is established under mixing-type assumptions. These assumptions are mild such that they include a wide range of time series models including auto-regressive models. Compared to independent observations, established convergence rates have additional logarithmic factors to compensate for additional complexity due to dependence among data points. The theoretical results are supported via various numerical simulation settings as well as an application to a macroeconomic data set.