mlsanityData to 14 Oct 2026

How Different from the Past? Spatio-Temporal Time Series Forecasting with Self-Supervised Deviation Learning

arXiv 2510.04908 ↗First posted 6 Oct 2025cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Spatio-temporal forecasting is essential for real-world applications such as traffic management and urban computing. Although recent methods have shown improved accuracy, they often fail to account for dynamic deviations between current inputs and historical patterns. These deviations contain critical signals that can significantly affect model performance. To fill this gap, we propose ST-SSDL, a Spatio-Temporal time series forecasting framework that incorporates a Self-Supervised Deviation Learning scheme to capture and utilize such deviations. ST-SSDL anchors each input to its historical average and discretizes the latent space using learnable prototypes that represent typical spatio-temporal patterns. Two auxiliary objectives are proposed to refine this structure: a contrastive loss that enhances inter-prototype discriminability and a deviation loss that regularizes the distance consistency between input representations and corresponding prototypes to quantify deviation. Optimized jointly with the forecasting objective, these components guide the model to organize its hidden space and improve generalization across diverse input conditions. Experiments on six benchmark datasets show that ST-SSDL consistently outperforms state-of-the-art baselines across multiple metrics. Visualizations further demonstrate its ability to adaptively respond to varying levels of deviation in complex spatio-temporal scenarios. Our code and datasets are available at https://github.com/Jimmy-7664/ST-SSDL.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 53 of its 88 references that are in the map. First posted on arXiv 6 Oct 2025.

Builds on

53 of 88 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  9. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  10. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  11. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  12. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
38 more
  1. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,136
  2. 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 829
  3. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 566
  4. 2020Adaptive Graph Convolutional Recurrent Network for Traffic Forecastingcited by 530
  5. 2021Spectral Temporal Graph Neural Network for Multivariate Time-series Forecastingcited by 299
  6. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 214
  7. 2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 180
  8. 2021Discrete Graph Structure Learning for Forecasting Multiple Time Seriescited by 171
  9. 2021Dynamic Graph Convolutional Recurrent Network for Traffic Prediction: Benchmark and Solutioncited by 157
  10. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
  11. 2022Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecastingcited by 151
  12. 2023PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Predictioncited by 137
  13. 2022Decoupled Dynamic Spatial-Temporal Graph Neural Network for Traffic Forecastingcited by 123
  14. 2022Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecastingcited by 117
  15. 2024WavLLM: Towards Robust and Adaptive Speech Large Language Modelcited by 109
  16. 2022Spatio-Temporal Meta-Graph Learning for Traffic Forecastingcited by 88
  17. 2022Spatio-Temporal Self-Supervised Learning for Traffic Flow Predictioncited by 77
  18. 2021DL-Traff: Survey and Benchmark of Deep Learning Models for Urban Traffic Predictioncited by 73
  19. 2022Towards Spatio-Temporal Aware Traffic Time Series Forecasting--Full Versioncited by 59
  20. 2023TimesURL: Self-supervised Contrastive Learning for Universal Time Series Representation Learningcited by 52
  21. 2024From Similarity to Superiority: Channel Clustering for Time Series Forecastingcited by 37
  22. 2023Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learningcited by 34
  23. 2023Spatial-Temporal-Decoupled Masked Pre-training for Spatiotemporal Forecastingcited by 27
  24. 2021Historical Inertia: A Neglected but Powerful Baseline for Long Sequence Time-series Forecastingcited by 26
  25. 2024Efficient Large-Scale Traffic Forecasting with Transformers: A Spatial Data Management Perspectivecited by 26
  26. 2023Automated Spatio-Temporal Graph Contrastive Learningcited by 24
  27. 2024STG-Mamba: Spatial-Temporal Graph Learning via Selective State Space Modelcited by 24
  28. 2023ST-MLP: A Cascaded Spatio-Temporal Linear Framework with Channel-Independence Strategy for Traffic Forecastingcited by 22
  29. 2021A Multi-view Multi-task Learning Framework for Multi-variate Time Series Forecastingcited by 21
  30. 2023Disentangling Structured Components: Towards Adaptive, Interpretable and Scalable Time Series Forecastingcited by 19
  31. 2025Spatiotemporal-aware Trend-Seasonality Decomposition Network for Traffic Flow Forecastingcited by 19
  32. 2021Event-Aware Multimodal Mobility Nowcastingcited by 18
  33. 2024TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Expertscited by 18
  34. 2024Heterogeneity-Informed Meta-Parameter Learning for Spatiotemporal Time Series Forecastingcited by 18
  35. 2024DyG-Mamba: Continuous State Space Modeling on Dynamic Graphscited by 17
  36. 2024Self-Supervised Learning of Time Series Representation via Diffusion Process and Imputation-Interpolation-Forecasting Maskcited by 10
  37. 2024AutoSTF: Decoupled Neural Architecture Search for Cost-Effective Automated Spatio-Temporal Forecastingcited by 10
  38. 2024Multi-Modality Spatio-Temporal Forecasting via Self-Supervised Learningcited by 9

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.