# Benign Overfitting in Time Series Linear Models with Over-Parameterization

arXiv: https://arxiv.org/abs/2204.08369 · HTML: https://mlsanity.com/paper/2204.08369/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 99 references that are in the map.
- First posted on arXiv 18 Apr 2022.

## Builds on (most cited first, 16 of 16)

- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [Just Interpolate: Kernel "Ridgeless" Regression Can Generalize](https://mlsanity.com/paper/1808.00387/) (2018)
- [Benign overfitting in ridge regression](https://mlsanity.com/paper/2009.14286/) (2020)
- [High-Dimensional Asymptotics of Prediction: Ridge Regression and Classification](https://mlsanity.com/paper/1507.03003/) (2015)
- [Regularized estimation in sparse high-dimensional time series models](https://mlsanity.com/paper/1311.4175/) (2013)
- [Learning from weakly dependent data under Dobrushin's condition](https://mlsanity.com/paper/1906.09247/) (2019)
- [Oracle Inequalities for High Dimensional Vector Autoregressions](https://mlsanity.com/paper/1311.0811/) (2013)
- [Regression from Dependent Observations](https://mlsanity.com/paper/1905.03353/) (2019)
- [Statistical Estimation from Dependent Data](https://mlsanity.com/paper/2107.09773/) (2021)
- [A Direct Estimation of High Dimensional Stationary Vector Autoregressions](https://mlsanity.com/paper/1307.0293/) (2013)
- [High Dimensional and Banded Vector Autoregressions](https://mlsanity.com/paper/1502.07831/) (2015)
- [Regularized Estimation of High-Dimensional Vector AutoRegressions with Weakly Dependent Innovations](https://mlsanity.com/paper/1912.09002/) (2019)
- [A note on strong-consistency of componentwise ARH(1) predictors](https://mlsanity.com/paper/1709.04938/) (2017)
- [Ratio-consistent estimation for long range dependent Toeplitz covariance with application to matrix data whitening](https://mlsanity.com/paper/2006.02070/) (2020)

## Built on by (most cited first, 4 of 4)

- [Asymptotics of Linear Regression with Linearly Dependent Data](https://mlsanity.com/paper/2412.03702/) (2024)
- [Generalisation and benign over-fitting for linear regression onto random functional covariates](https://mlsanity.com/paper/2508.13895/) (2025)
- [Bayesian Analysis for Over-parameterized Linear Model via Effective Spectra](https://mlsanity.com/paper/2305.15754/) (2023)
- [Bayesian Inference for Consistent Predictions in Overparameterized Nonlinear Regression](https://mlsanity.com/paper/2404.04498/) (2024)

## Abstract (arXiv)

The success of large-scale models in recent years has increased the importance of statistical models with numerous parameters. Several studies have analyzed over-parameterized linear models with high-dimensional data, which may not be sparse; however, existing results rely on the assumption of sample independence. In this study, we analyze a linear regression model with dependent time-series data in an over-parameterized setting. We consider an estimator using interpolation and develop a theory for the excess risk of the estimator. Then, we derive non-asymptotic risk bounds for the estimator for cases with dependent data. This analysis reveals that the coherence of the temporal covariance plays a key role; the risk bound is influenced by the product of temporal covariance matrices at different time steps. Moreover, we show the convergence rate of the risk bound and demonstrate that it is also influenced by the coherence of the temporal covariance. Finally, we provide several examples of specific dependent processes applicable to our setting.