# Temporal Pattern Attention for Multivariate Time Series Forecasting

arXiv: https://arxiv.org/abs/1809.04206 · HTML: https://mlsanity.com/paper/1809.04206/

## Key facts

- Cited by 125 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2018.

## Built on by (most cited first, 30 of 125)

- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction](https://mlsanity.com/paper/2106.09305/) (2021)
- [Discrete Graph Structure Learning for Forecasting Multiple Time Series](https://mlsanity.com/paper/2101.06861/) (2021)
- [Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures](https://mlsanity.com/paper/2207.01186/) (2022)
- [A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection](https://mlsanity.com/paper/2307.03759/) (2023)
- [Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement](https://mlsanity.com/paper/2301.03028/) (2023)
- [Spatio-Temporal Meta-Graph Learning for Traffic Forecasting](https://mlsanity.com/paper/2211.14701/) (2022)
- [DL-Traff: Survey and Benchmark of Deep Learning Models for Urban Traffic Prediction](https://mlsanity.com/paper/2108.09091/) (2021)
- [Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs](https://mlsanity.com/paper/2202.08408/) (2022)
- [Forecast Evaluation for Data Scientists: Common Pitfalls and Best Practices](https://mlsanity.com/paper/2203.10716/) (2022)
- [A Temporal Kolmogorov-Arnold Transformer for Time Series Forecasting](https://mlsanity.com/paper/2406.02486/) (2024)
- [AutoCTS: Automated Correlated Time Series Forecasting -- Extended Version](https://mlsanity.com/paper/2112.11174/) (2021)
- [Long-Range Transformers for Dynamic Spatiotemporal Forecasting](https://mlsanity.com/paper/2109.12218/) (2021)
- [TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders](https://mlsanity.com/paper/2303.00320/) (2023)
- [ScoreGrad: Multivariate Probabilistic Time Series Forecasting with Continuous Energy-based Generative Models](https://mlsanity.com/paper/2106.10121/) (2021)
- [Multi-Scale Adaptive Graph Neural Network for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2201.04828/) (2022)
- [TACTiS: Transformer-Attentional Copulas for Time Series](https://mlsanity.com/paper/2202.03528/) (2022)
- [Learning the Evolutionary and Multi-scale Graph Structure for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2206.13816/) (2022)
- [Historical Inertia: A Neglected but Powerful Baseline for Long Sequence Time-series Forecasting](https://mlsanity.com/paper/2103.16349/) (2021)
- [TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series](https://mlsanity.com/paper/2310.01327/) (2023)
- [Merlion: A Machine Learning Library for Time Series](https://mlsanity.com/paper/2109.09265/) (2021)
- [Spatio-temporal graph neural networks for multi-site PV power forecasting](https://mlsanity.com/paper/2107.13875/) (2021)
- [Joint Neural Architecture and Hyperparameter Search for Correlated Time Series Forecasting](https://mlsanity.com/paper/2211.16126/) (2022)
- [Multivariate Time Series Forecasting with Latent Graph Inference](https://mlsanity.com/paper/2203.03423/) (2022)
- [NuTime: Numerically Multi-Scaled Embedding for Large-Scale Time-Series Pretraining](https://mlsanity.com/paper/2310.07402/) (2023)
- [Multi-scale Attention Flow for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2205.07493/) (2022)
- [Adjusting for Autocorrelated Errors in Neural Networks for Time Series](https://mlsanity.com/paper/2101.12578/) (2021)
- [LightCTS: A Lightweight Framework for Correlated Time Series Forecasting](https://mlsanity.com/paper/2302.11974/) (2023)
- [EpiGNN: Exploring Spatial Transmission with Graph Neural Network for Regional Epidemic Forecasting](https://mlsanity.com/paper/2208.11517/) (2022)
- [pTSE: A Multi-model Ensemble Method for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2305.11304/) (2023)

## Abstract (arXiv)

Forecasting multivariate time series data, such as prediction of electricity consumption, solar power production, and polyphonic piano pieces, has numerous valuable applications. However, complex and non-linear interdependencies between time steps and series complicate the task. To obtain accurate prediction, it is crucial to model long-term dependency in time series data, which can be achieved to some good extent by recurrent neural network (RNN) with attention mechanism. Typical attention mechanism reviews the information at each previous time step and selects the relevant information to help generate the outputs, but it fails to capture the temporal patterns across multiple time steps. In this paper, we propose to use a set of filters to extract time-invariant temporal patterns, which is similar to transforming time series data into its "frequency domain". Then we proposed a novel attention mechanism to select relevant time series, and use its "frequency domain" information for forecasting. We applied the proposed model on several real-world tasks and achieved state-of-the-art performance in all of them with only one exception.