# Parallel Spatio-Temporal Attention-Based TCN for Multivariate Time Series Prediction

arXiv: https://arxiv.org/abs/2203.00971 · HTML: https://mlsanity.com/paper/2203.00971/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [Regularizing and Optimizing LSTM Language Models](https://mlsanity.com/paper/1708.02182/) (2017)
- [Neural Machine Translation in Linear Time](https://mlsanity.com/paper/1610.10099/) (2016)
- [A Convolutional Encoder Model for Neural Machine Translation](https://mlsanity.com/paper/1611.02344/) (2016)
- [Deep Learning for Community Detection: Progress, Challenges and Opportunities](https://mlsanity.com/paper/2005.08225/) (2020)
- [Deep Learning with Long Short-Term Memory for Time Series Prediction](https://mlsanity.com/paper/1810.10161/) (2018)
- [Distributed and parallel time series feature extraction for industrial big data applications](https://mlsanity.com/paper/1610.07717/) (2016)
- [EA-LSTM: Evolutionary Attention-based LSTM for Time Series Prediction](https://mlsanity.com/paper/1811.03760/) (2018)
- [DSTP-RNN: a dual-stage two-phase attention-based recurrent neural networks for long-term and multivariate time series prediction](https://mlsanity.com/paper/1904.07464/) (2019)
- [Temporal Convolutional Attention-based Network For Sequence Modeling](https://mlsanity.com/paper/2002.12530/) (2020)

## Built on by (most cited first, 12 of 12)

- [Leveraging TCN and Transformer for effective visual-audio fusion in continuous emotion recognition](https://mlsanity.com/paper/2303.08356/) (2023)
- [VideoScan: Enabling Efficient Streaming Video Understanding via Frame-level Semantic Carriers](https://mlsanity.com/paper/2503.09387/) (2025)
- [Boosting Continuous Emotion Recognition with Self-Pretraining using Masked Autoencoders, Temporal Convolutional Networks, and Transformers](https://mlsanity.com/paper/2403.11440/) (2024)
- [Emotion Recognition with CLIP and Sequential Learning](https://mlsanity.com/paper/2503.09929/) (2025)
- [HalluRNN: Mitigating Hallucinations via Recurrent Cross-Layer Reasoning in Large Vision-Language Models](https://mlsanity.com/paper/2506.17587/) (2025)
- [Mlinear: Rethink the Linear Model for Time-series Forecasting](https://mlsanity.com/paper/2305.04800/) (2023)
- [A Predictive Model Based on Transformer with Statistical Feature Embedding in Manufacturing Sensor Dataset](https://mlsanity.com/paper/2407.06682/) (2024)
- [Spatiotemporal-Linear: Towards Universal Multivariate Time Series Forecasting](https://mlsanity.com/paper/2312.14869/) (2023)
- [Performance Evaluation of Deep Learning Models for Water Quality Index Prediction: A Comparative Study of LSTM, TCN, ANN, and MLP](https://mlsanity.com/paper/2411.01527/) (2024)
- [Intelligent Collaborative Optimization for Rubber Tyre Film Production Based on Multi-path Differentiated Clipping Proximal Policy Optimization](https://mlsanity.com/paper/2511.12060/) (2025)
- [Wavelet-Aware Anomaly Detection in Multi-Channel User Logs via Deviation Modulation and Resolution-Adaptive Attention](https://mlsanity.com/paper/2601.12231/) (2026)
- [Generative AI-enhanced Probabilistic Multi-Fidelity Surrogate Modeling Via Transfer Learning](https://mlsanity.com/paper/2602.00072/) (2026)

## Abstract (arXiv)

As industrial systems become more complex and monitoring sensors for everything from surveillance to our health become more ubiquitous, multivariate time series prediction is taking an important place in the smooth-running of our society. A recurrent neural network with attention to help extend the prediction windows is the current-state-of-the-art for this task. However, we argue that their vanishing gradients, short memories, and serial architecture make RNNs fundamentally unsuited to long-horizon forecasting with complex data. Temporal convolutional networks (TCNs) do not suffer from gradient problems and they support parallel calculations, making them a more appropriate choice. Additionally, they have longer memories than RNNs, albeit with some instability and efficiency problems. Hence, we propose a framework, called PSTA-TCN, that combines a parallel spatio-temporal attention mechanism to extract dynamic internal correlations with stacked TCN backbones to extract features from different window sizes. The framework makes full use parallel calculations to dramatically reduce training times, while substantially increasing accuracy with stable prediction windows up to 13 times longer than the status quo.