mlsanityData to 7 Oct 2026

Spatial-Temporal-Decoupled Masked Pre-training for Spatiotemporal Forecasting

arXiv 2312.00516 ↗First posted 1 Dec 2023cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Spatiotemporal forecasting techniques are significant for various domains such as transportation, energy, and weather. Accurate prediction of spatiotemporal series remains challenging due to the complex spatiotemporal heterogeneity. In particular, current end-to-end models are limited by input length and thus often fall into spatiotemporal mirage, i.e., similar input time series followed by dissimilar future values and vice versa. To address these problems, we propose a novel self-supervised pre-training framework Spatial-Temporal-Decoupled Masked Pre-training (STD-MAE) that employs two decoupled masked autoencoders to reconstruct spatiotemporal series along the spatial and temporal dimensions. Rich-context representations learned through such reconstruction could be seamlessly integrated by downstream predictors with arbitrary architectures to augment their performances. A series of quantitative and qualitative evaluations on six widely used benchmarks (PEMS03, PEMS04, PEMS07, PEMS08, METR-LA, and PEMS-BAY) are conducted to validate the state-of-the-art performance of STD-MAE. Codes are available at https://github.com/Jimmy-7664/STD-MAE.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 28 of its 42 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

28 of 42 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  6. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  7. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  8. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  9. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  10. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  11. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,125
  12. 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 820
16 more
  1. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 563
  2. 2020Adaptive Graph Convolutional Recurrent Network for Traffic Forecastingcited by 525
  3. 2021Spectral Temporal Graph Neural Network for Multivariate Time-series Forecastingcited by 298
  4. 2020Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecastingcited by 212
  5. 2021Spatial-Temporal Graph ODE Networks for Traffic Flow Forecastingcited by 209
  6. 2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 179
  7. 2022Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecastingcited by 149
  8. 2023PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Predictioncited by 135
  9. 2022Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecastingcited by 117
  10. 2023Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysiscited by 113
  11. 2022Spatio-Temporal Meta-Graph Learning for Traffic Forecastingcited by 88
  12. 2021DL-Traff: Survey and Benchmark of Deep Learning Models for Urban Traffic Predictioncited by 73
  13. 2021Z-GCNETs: Time Zigzags at Graph Convolutional Networks for Time Series Forecastingcited by 67
  14. 2023Ti-MAE: Self-Supervised Masked Time Series Autoencoderscited by 63
  15. 2022Towards Spatio-Temporal Aware Traffic Time Series Forecasting--Full Versioncited by 59
  16. 2021A Multi-view Multi-task Learning Framework for Multi-variate Time Series Forecastingcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 11; 2026: 13.

27 papers.

Sort:

By area:

  1. 2025Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Reviewcited by 10
  2. 2024ST-LoRA: Low-rank Adaptation for Spatio-Temporal Forecastingcited by 9
  3. 2025Message-Passing State-Space Models: Improving Graph Learning with Modern Sequence Modelingcited by 9
  4. 2025Decoupling Spatio-Temporal Prediction: When Lightweight Large Models Meet Adaptive Hypergraphscited by 4
  5. 2025How Different from the Past? Spatio-Temporal Time Series Forecasting with Self-Supervised Deviation Learningcited by 4
  6. 2024Cross Space and Time: A Spatio-Temporal Unitized Model for Traffic Flow Forecastingcited by 3
  7. 2025Persistent Homology-induced Graph Ensembles for Time Series Regressionscited by 3
  8. 2026A General ReLearner: Empowering Spatiotemporal Prediction by Re-learning Input-label Residualcited by 1
  9. 2026UrbanFM: Scaling Urban Spatio-Temporal Foundation Modelscited by 1
  10. 2026Dynamic Sheaf Diffusion Networks with Adaptive Local Structure for Heterogeneous Spatio-Temporal Graph Learningcited by 1
  11. 2026GAMMA-Net: Adaptive Long-Horizon Traffic Spatio-Temporal Forecasting Model based on Interleaved Graph Attention and Multi-Axis Mambacited by 1
  12. 2026Net-Ev2: A Generative Simulator for Network Event Evolutioncited by 1
15 more
  1. 2024HiMoE: Heterogeneity-Informed Mixture-of-Experts for Fair Spatial-Temporal Forecastingcited by 0
  2. 2025Embedding spatial context in urban traffic forecasting with contrastive pre-trainingcited by 0
  3. 2025Efficient Traffic State Prediction With Dynamic Joint Spatio-Temporal Relation Inferencecited by 0
  4. 2025Lightweight and Interpretable Transformer via Mixed Graph Algorithm Unrolling for Traffic Forecastcited by 0
  5. 2025SPAR: Self-supervised Placement-Aware Representation Learning for Distributed Sensingcited by 0
  6. 2025Proxy Reconstruction Pre-training for Ramp Flow Prediction at Highway Interchangescited by 0
  7. 2025Efficient Linear Attention for Multivariate Time Series Modeling via Entropy Equalitycited by 0
  8. 2026UniST-Pred: A Robust Unified Framework for Spatio-Temporal Traffic Forecasting in Transportation Networks Under Disruptionscited by 0
  9. 2026CoMemNet: A Continual Memory Network with Drift-Aware Sampling for Traffic Predictioncited by 0
  10. 2026Dimensional Balance Improves Large Scale Spatiotemporal Prediction Performancecited by 0
  11. 2026DSFNet: Learning Dual-Domain Spectral Operators for Multi-Modality Spatio-Temporal Forecasting in Urban Transportation Systemscited by 0
  12. 2026MP3: Multi-Period Pattern Pre-training for Spatio-Temporal Forecastingcited by 0
  13. 2026Global-Scale Self-Supervised Spatiotemporal Learning for NDVI Time-Series Reconstructioncited by 0
  14. 2026ButterMamba: Butterworth-Enhanced Spatial-Temporal Mamba for Efficient Traffic Flow Predictioncited by 0
  15. 2026TimeES: Probabilistic and Deterministic Time Series Forecasting via Evolutionary Spectracited by 0