# DSTP-RNN: a dual-stage two-phase attention-based recurrent neural networks for long-term and multivariate time series prediction

arXiv: https://arxiv.org/abs/1904.07464 · HTML: https://mlsanity.com/paper/1904.07464/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2019.

## Built on by (most cited first, 20 of 20)

- [Fully-Connected Spatial-Temporal Graph for Multivariate Time-Series Data](https://mlsanity.com/paper/2309.05305/) (2023)
- [Parallel Spatio-Temporal Attention-Based TCN for Multivariate Time Series Prediction](https://mlsanity.com/paper/2203.00971/) (2022)
- [Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology](https://mlsanity.com/paper/2310.05227/) (2023)
- [ODformer: Spatial-Temporal Transformers for Long Sequence Origin-Destination Matrix Forecasting Against Cross Application Scenario](https://mlsanity.com/paper/2208.08218/) (2022)
- [TSEM: Temporally Weighted Spatiotemporal Explainable Neural Network for Multivariate Time Series](https://mlsanity.com/paper/2205.13012/) (2022)
- [DGCformer: Deep Graph Clustering Transformer for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2405.08440/) (2024)
- [Graph Generation Powered with LLMs for Boosting Multivariate Time-Series Representation Learning](https://mlsanity.com/paper/2403.03645/) (2024)
- [A Novel Hybrid Approach Using an Attention-Based Transformer + GRU Model for Predicting Cryptocurrency Prices](https://mlsanity.com/paper/2504.17079/) (2025)
- [TimeSense:Making Large Language Models Proficient in Time-Series Analysis](https://mlsanity.com/paper/2511.06344/) (2025)
- [Counterfactual Explainable AI (XAI) Method for Deep Learning-Based Multivariate Time Series Classification](https://mlsanity.com/paper/2511.13237/) (2025)
- [A Comprehensive Survey on Deep Neural Image Deblurring](https://mlsanity.com/paper/2310.04719/) (2023)
- [Non-stationary and Sparsely-correlated Multi-output Gaussian Process with Spike-and-Slab Prior](https://mlsanity.com/paper/2409.03149/) (2024)
- [Visual Sensation and Perception Computational Models for Deep Learning: State of the art, Challenges and Prospects](https://mlsanity.com/paper/2109.03391/) (2021)
- [HigeNet: A Highly Efficient Modeling for Long Sequence Time Series Prediction in AIOps](https://mlsanity.com/paper/2211.07642/) (2022)
- [Time Series Forecasting via Semi-Asymmetric Convolutional Architecture with Global Atrous Sliding Window](https://mlsanity.com/paper/2301.13691/) (2023)
- [GenFormer: A Deep-Learning-Based Approach for Generating Multivariate Stochastic Processes](https://mlsanity.com/paper/2402.02010/) (2024)
- [Multiscale autonomous forecasting of plasma systems' dynamics using neural networks](https://mlsanity.com/paper/2502.11203/) (2025)
- [PHEATPRUNER: Interpretable Data-centric Feature Selection for Multivariate Time Series Classification through Persistent Homology](https://mlsanity.com/paper/2504.18329/) (2025)
- [A Review of the Long Horizon Forecasting Problem in Time Series Analysis](https://mlsanity.com/paper/2506.12809/) (2025)
- [Proportional integral derivative booster for neural networks-based time-series prediction: Case of water demand prediction](https://mlsanity.com/paper/2512.06357/) (2025)

## Abstract (arXiv)

Long-term prediction of multivariate time series is still an important but challenging problem. The key to solve this problem is to capture the spatial correlations at the same time, the spatio-temporal relationships at different times and the long-term dependence of the temporal relationships between different series. Attention-based recurrent neural networks (RNN) can effectively represent the dynamic spatio-temporal relationships between exogenous series and target series, but it only performs well in one-step time prediction and short-term time prediction. In this paper, inspired by human attention mechanism including the dual-stage two-phase (DSTP) model and the influence mechanism of target information and non-target information, we propose DSTP-based RNN (DSTP-RNN) and DSTP-RNN-2 respectively for long-term time series prediction. Specifically, we first propose the DSTP-based structure to enhance the spatial correlations between exogenous series. The first phase produces violent but decentralized response weight, while the second phase leads to stationary and concentrated response weight. Secondly, we employ multiple attentions on target series to boost the long-term dependence. Finally, we study the performance of deep spatial attention mechanism and provide experiment and interpretation. Our methods outperform nine baseline methods on four datasets in the fields of energy, finance, environment and medicine, respectively.