mlsanityData to 7 Oct 2026

Learning to Factorize and Adapt: A Versatile Approach Toward Universal Spatio-Temporal Foundation Models

arXiv 2601.12083 ↗First posted 17 Jan 2026cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Spatio-Temporal (ST) Foundation Models (STFMs) promise cross-dataset generalization, yet joint ST pretraining is computationally expensive and grapples with the heterogeneity of domain-specific spatial patterns. Substantially extending our preliminary conference version, we present FactoST-v2, an enhanced factorized framework redesigned for full weight transfer and arbitrary-length generalization. FactoST-v2 decouples universal temporal learning from domain-specific spatial adaptation. The first stage pretrains a minimalist encoder-only backbone using randomized sequence masking to capture invariant temporal dynamics, enabling probabilistic quantile prediction across variable horizons. The second stage employs a streamlined adapter to rapidly inject spatial awareness via meta adaptive learning and prompting. Comprehensive evaluations across diverse domains demonstrate that FactoST-v2 achieves state-of-the-art accuracy with linear efficiency - significantly outperforming existing foundation models in zero-shot and few-shot scenarios while rivaling domain-specific expert baselines. This factorized paradigm offers a practical, scalable path toward truly universal STFMs. Code is available at https://github.com/CityMind-Lab/FactoST.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 74 of its 121 references that are in the map. First posted on arXiv 17 Jan 2026.

Builds on

74 of 121 references.

References · 74

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  6. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  7. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  8. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  9. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  10. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,125
  11. 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 820
  12. 2024Chronos: Learning the Language of Time Seriescited by 712
38 more
  1. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  2. 2023A decoder-only foundation model for time-series forecastingcited by 650
  3. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  4. 2020Adaptive Graph Convolutional Recurrent Network for Traffic Forecastingcited by 525
  5. 2018T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Predictioncited by 399
  6. 2024TimeMixer: Decomposable Multiscale Mixing for Time Series Forecastingcited by 333
  7. 2016Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Predictioncited by 330
  8. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
  9. 2024TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variablescited by 241
  10. 2017A Multi-Horizon Quantile Recurrent Forecastercited by 211
  11. 2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 179
  12. 2019Deep Learning for Spatio-Temporal Data Mining: A Surveycited by 171
  13. 2021Discrete Graph Structure Learning for Forecasting Multiple Time Seriescited by 169
  14. 2022Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecastingcited by 149
  15. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  16. 2023Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Surveycited by 140
  17. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 139
  18. 2022Decoupled Dynamic Spatial-Temporal Graph Neural Network for Traffic Forecastingcited by 122
  19. 2022Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecastingcited by 117
  20. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 111
  21. 2024UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Predictioncited by 86
  22. 2024Spatial-Temporal Large Language Model for Traffic Predictioncited by 81
  23. 2024UrbanGPT: Spatio-Temporal Large Language Modelscited by 81
  24. 2022Spatio-Temporal Self-Supervised Learning for Traffic Flow Predictioncited by 77
  25. 2024Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspectivecited by 75
  26. 2022AirFormer: Predicting Nationwide Air Quality in China with Transformerscited by 73
  27. 2021A Review of Graph Neural Networks and Their Applications in Power Systemscited by 62
  28. 2025Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecastingcited by 59
  29. 2024Deep Learning for Multivariate Time Series Imputation: A Surveycited by 57
  30. 2022Spatio-Temporal Gating-Adjacency GCN for Human Motion Predictioncited by 48
  31. 2021When Do Contrastive Learning Signals Help Spatio-Temporal Graph Forecasting?cited by 36
  32. 2020Generative Adversarial Networks for Spatio-temporal Data: A Surveycited by 35
  33. 2024Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfercited by 31
  34. 2024OpenCity: Open Spatio-Temporal Foundation Models for Traffic Predictioncited by 27
  35. 2024Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecastingcited by 26
  36. 2024CityBench: Evaluating the Capabilities of Large Language Models for Urban Taskscited by 26
  37. 2024Air Quality Prediction with Physics-Guided Dual Neural ODEs in Open Systemscited by 25
  38. 2023Spatio-Temporal Graph Neural Networks: A Surveycited by 23

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026WPBench: A Comprehensive Benchmark for Wind Power Forecastingcited by 2
  2. 2026UrbanFM: Scaling Urban Spatio-Temporal Foundation Modelscited by 1