# Adjusting for Autocorrelated Errors in Neural Networks for Time Series

arXiv: https://arxiv.org/abs/2101.12578 · HTML: https://mlsanity.com/paper/2101.12578/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 85 references that are in the map.
- First posted on arXiv 28 Jan 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting](https://mlsanity.com/paper/2007.02842/) (2020)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [Coordinate Descent Algorithms](https://mlsanity.com/paper/1502.04759/) (2015)
- [A Multi-Horizon Quantile Recurrent Forecaster](https://mlsanity.com/paper/1711.11053/) (2017)
- [Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/1905.03806/) (2019)
- [High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes](https://mlsanity.com/paper/1910.03002/) (2019)
- [Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows](https://mlsanity.com/paper/2002.06103/) (2020)
- [Temporal Pattern Attention for Multivariate Time Series Forecasting](https://mlsanity.com/paper/1809.04206/) (2018)
- [Multivariate LSTM-FCNs for Time Series Classification](https://mlsanity.com/paper/1801.04503/) (2018)
- [Deep Learning for Time-Series Analysis](https://mlsanity.com/paper/1701.01887/) (2017)
- [Integrated Pre-Processing for Bayesian Nonlinear System Identification with Gaussian Processes](https://mlsanity.com/paper/1303.2912/) (2013)
- [Seq2Tens: An Efficient Representation of Sequences by Low-Rank Tensor Projections](https://mlsanity.com/paper/2006.07027/) (2020)

## Built on by (most cited first, 18 of 18)

- [Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency](https://mlsanity.com/paper/2206.08496/) (2022)
- [SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling](https://mlsanity.com/paper/2302.00861/) (2023)
- [Nominality Score Conditioned Time Series Anomaly Detection by Point/Sequential Reconstruction](https://mlsanity.com/paper/2310.15416/) (2023)
- [PPGF: Probability Pattern-Guided Time Series Forecasting](https://mlsanity.com/paper/2502.12802/) (2025)
- [Sub-Adjacent Transformer: Improving Time Series Anomaly Detection with Reconstruction Error from Sub-Adjacent Neighborhoods](https://mlsanity.com/paper/2404.18948/) (2024)
- [Deep Learning for Time Series Forecasting: A Survey](https://mlsanity.com/paper/2503.10198/) (2025)
- [FreDo: Frequency Domain-based Long-Term Time Series Forecasting](https://mlsanity.com/paper/2205.12301/) (2022)
- [Multivariate Probabilistic Time Series Forecasting with Correlated Errors](https://mlsanity.com/paper/2402.01000/) (2024)
- [Better Batch for Deep Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2305.17028/) (2023)
- [Probabilistic Traffic Forecasting with Dynamic Regression](https://mlsanity.com/paper/2301.06650/) (2023)
- [Simple Feedfoward Neural Networks are Almost All You Need for Time Series Forecasting](https://mlsanity.com/paper/2503.23621/) (2025)
- [Optimal Adaptive Prediction Intervals for Electricity Load Forecasting in Distribution Systems via Reinforcement Learning](https://mlsanity.com/paper/2205.08698/) (2022)
- [Link Representation Learning for Probabilistic Travel Time Estimation](https://mlsanity.com/paper/2407.05895/) (2024)
- [RINS-T: Robust Implicit Neural Solvers for Time Series Linear Inverse Problems](https://mlsanity.com/paper/2510.17396/) (2025)
- [Correlation recurrent units: A novel neural architecture for improving the predictive performance of time-series data](https://mlsanity.com/paper/2211.16653/) (2022)
- [Symbol-Temporal Consistency Self-supervised Learning for Robust Time Series Classification](https://mlsanity.com/paper/2509.19654/) (2025)
- [Error Adjustment Based on Spatiotemporal Correlation Fusion for Traffic Forecasting](https://mlsanity.com/paper/2510.23656/) (2025)
- [Deep Time-Series Forecasting in 10 Years: A Survey](https://mlsanity.com/paper/2603.19899/) (2026)

## Abstract (arXiv)

An increasing body of research focuses on using neural networks to model time series. A common assumption in training neural networks via maximum likelihood estimation on time series is that the errors across time steps are uncorrelated. However, errors are actually autocorrelated in many cases due to the temporality of the data, which makes such maximum likelihood estimations inaccurate. In this paper, in order to adjust for autocorrelated errors, we propose to learn the autocorrelation coefficient jointly with the model parameters. In our experiments, we verify the effectiveness of our approach on time series forecasting. Results across a wide range of real-world datasets with various state-of-the-art models show that our method enhances performance in almost all cases. Based on these results, we suggest empirical critical values to determine the severity of autocorrelated errors. We also analyze several aspects of our method to demonstrate its advantages. Finally, other time series tasks are also considered to validate that our method is not restricted to only forecasting.