# When Do Contrastive Learning Signals Help Spatio-Temporal Graph Forecasting?

arXiv: https://arxiv.org/abs/2108.11873 · HTML: https://mlsanity.com/paper/2108.11873/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 48 references that are in the map.
- First posted on arXiv 26 Aug 2021.

## Builds on (most cited first, 29 of 29)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting](https://mlsanity.com/paper/2007.02842/) (2020)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://mlsanity.com/paper/2006.09963/) (2020)
- [Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction](https://mlsanity.com/paper/1610.00081/) (2016)
- [GPT-GNN: Generative Pre-Training of Graph Neural Networks](https://mlsanity.com/paper/2006.15437/) (2020)
- [Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting](https://mlsanity.com/paper/2103.07719/) (2021)
- [Graph Self-Supervised Learning: A Survey](https://mlsanity.com/paper/2103.00111/) (2021)
- [Self-supervised Co-training for Video Representation Learning](https://mlsanity.com/paper/2010.09709/) (2020)
- [Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2012.09641/) (2020)
- [Dynamic Graph Convolutional Recurrent Network for Traffic Prediction: Benchmark and Solution](https://mlsanity.com/paper/2104.14917/) (2021)
- [When Does Self-Supervision Help Graph Convolutional Networks?](https://mlsanity.com/paper/2006.09136/) (2020)
- [Contrastive Self-supervised Learning for Graph Classification](https://mlsanity.com/paper/2009.05923/) (2020)
- [Semi-Supervised Hierarchical Recurrent Graph Neural Network for City-Wide Parking Availability Prediction](https://mlsanity.com/paper/1911.10516/) (2019)
- [Iterative Graph Self-Distillation](https://mlsanity.com/paper/2010.12609/) (2020)

## Built on by (most cited first, 30 of 36)

- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey](https://mlsanity.com/paper/2303.14483/) (2023)
- [LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting](https://mlsanity.com/paper/2306.08259/) (2023)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction](https://mlsanity.com/paper/2402.11838/) (2024)
- [Contrastive Trajectory Similarity Learning with Dual-Feature Attention](https://mlsanity.com/paper/2210.05155/) (2022)
- [Towards Graph Contrastive Learning: A Survey and Beyond](https://mlsanity.com/paper/2405.11868/) (2024)
- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [Do We Really Need Graph Neural Networks for Traffic Forecasting?](https://mlsanity.com/paper/2301.12603/) (2023)
- [Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models](https://mlsanity.com/paper/2503.11411/) (2025)
- [A Survey of Generative Techniques for Spatial-Temporal Data Mining](https://mlsanity.com/paper/2405.09592/) (2024)
- [Traffic Forecasting on New Roads Using Spatial Contrastive Pre-Training (SCPT)](https://mlsanity.com/paper/2305.05237/) (2023)
- [Spatio-Temporal Meta Contrastive Learning](https://mlsanity.com/paper/2310.17678/) (2023)
- [Data Augmentation on Graphs: A Technical Survey](https://mlsanity.com/paper/2212.09970/) (2022)
- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [Self-Supervised Representation Learning for Geospatial Objects: A Survey](https://mlsanity.com/paper/2408.12133/) (2024)
- [STGformer: Efficient Spatiotemporal Graph Transformer for Traffic Forecasting](https://mlsanity.com/paper/2410.00385/) (2024)
- [Personalized Federated Learning for Spatio-Temporal Forecasting: A Dual Semantic Alignment-Based Contrastive Approach](https://mlsanity.com/paper/2404.03702/) (2024)
- [STS-CCL: Spatial-Temporal Synchronous Contextual Contrastive Learning for Urban Traffic Forecasting](https://mlsanity.com/paper/2307.02507/) (2023)
- [Revealing the Power of Masked Autoencoders in Traffic Forecasting](https://mlsanity.com/paper/2309.15169/) (2023)
- [SimPSI: A Simple Strategy to Preserve Spectral Information in Time Series Data Augmentation](https://mlsanity.com/paper/2312.05790/) (2023)
- [LSTTN: A Long-Short Term Transformer-based Spatio-temporal Neural Network for Traffic Flow Forecasting](https://mlsanity.com/paper/2403.16495/) (2024)
- [DualCast: A Model to Disentangle Aperiodic Events from Traffic Series](https://mlsanity.com/paper/2411.18286/) (2024)
- [Dynamic Graph Representation with Contrastive Learning for Financial Market Prediction: Integrating Temporal Evolution and Static Relations](https://mlsanity.com/paper/2412.04034/) (2024)
- [PreMixer: MLP-Based Pre-training Enhanced MLP-Mixers for Large-scale Traffic Forecasting](https://mlsanity.com/paper/2412.13607/) (2024)
- [Fusing Cross-Domain Knowledge from Multimodal Data to Solve Problems in the Physical World](https://mlsanity.com/paper/2506.03155/) (2025)
- [Learning to Factorize and Adapt: A Versatile Approach Toward Universal Spatio-Temporal Foundation Models](https://mlsanity.com/paper/2601.12083/) (2026)
- [TEAFormers: TEnsor-Augmented Transformers for Multi-Dimensional Time Series Forecasting](https://mlsanity.com/paper/2410.20439/) (2024)
- [Enhancing Graph Neural Networks in Large-scale Traffic Incident Analysis with Concurrency Hypothesis](https://mlsanity.com/paper/2411.02542/) (2024)

## Abstract (arXiv)

Deep learning models are modern tools for spatio-temporal graph (STG) forecasting. Though successful, we argue that data scarcity is a key factor limiting their recent improvements. Meanwhile, contrastive learning has been an effective method for providing self-supervision signals and addressing data scarcity in various domains. In view of this, one may ask: can we leverage the additional signals from contrastive learning to alleviate data scarcity, so as to benefit STG forecasting? To answer this question, we present the first systematic exploration on incorporating contrastive learning into STG forecasting. Specifically, we first elaborate two potential schemes for integrating contrastive learning. We then propose two feasible and efficient designs of contrastive tasks that are performed on the node or graph level. The empirical study on STG benchmarks demonstrates that integrating graph-level contrast with the joint learning scheme achieves the best performance. In addition, we introduce four augmentations for STG data, which perturb the data in terms of graph structure, time domain, and frequency domain. Experimental results reveal that the model is not sensitive to the proposed augmentations' semantics. Lastly, we extend the classic contrastive loss via a rule-based strategy that filters out the most semantically similar negatives, yielding performance gains. We also provide explanations and insights based on the above experimental findings. Code is available at https://github.com/liuxu77/STGCL.