mlsanityData to 7 Oct 2026

TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts

arXiv 2403.02600 ↗First posted 5 Mar 2024cs.LG cs.SIcited by 18

On the map: Mixture-of-experts → MoE for time series and graphs

Abstract (arXiv)

Accurate traffic forecasting is challenging due to the complex dependency on road networks, various types of roads, and the abrupt speed change due to the events. Recent works mainly focus on dynamic spatial modeling with adaptive graph embedding or graph attention having less consideration for temporal characteristics and in-situ modeling. In this paper, we propose a novel deep learning model named TESTAM, which individually models recurring and non-recurring traffic patterns by a mixture-of-experts model with three experts on temporal modeling, spatio-temporal modeling with static graph, and dynamic spatio-temporal dependency modeling with dynamic graph. By introducing different experts and properly routing them, TESTAM could better model various circumstances, including spatially isolated nodes, highly related nodes, and recurring and non-recurring events. For the proper routing, we reformulate a gating problem into a classification problem with pseudo labels. Experimental results on three public traffic network datasets, METR-LA, PEMS-BAY, and EXPY-TKY, demonstrate that TESTAM achieves a better indication and modeling of recurring and non-recurring traffic. We published the official code at https://github.com/HyunWookL/TESTAM

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 25 of its 35 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

25 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Spatial Mixture-of-Experts24 Nov

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,125
  6. 2016HyperNetworkscited by 1,003
  7. 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 820
  8. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  9. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  10. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 563
  11. 2020Adaptive Graph Convolutional Recurrent Network for Traffic Forecastingcited by 525
  12. 2019GMAN: A Graph Multi-Attention Network for Traffic Predictioncited by 433
12 more
  1. 2021Spectral Temporal Graph Neural Network for Multivariate Time-series Forecastingcited by 298
  2. 2013Learning Factored Representations in a Deep Mixture of Expertscited by 275
  3. 2019Time2Vec: Learning a Vector Representation of Timecited by 208
  4. 2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 179
  5. 2021Discrete Graph Structure Learning for Forecasting Multiple Time Seriescited by 169
  6. 2017Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learningcited by 125
  7. 2022Spatio-Temporal Meta-Graph Learning for Traffic Forecastingcited by 88
  8. 2020Coupled Layer-wise Graph Convolution for Transportation Demand Predictioncited by 54
  9. 2017Deciding How to Decide: Dynamic Routing in Artificial Neural Networkscited by 33
  10. 2021Learning to Remember Patterns: Pattern Matching Memory Networks for Traffic Forecastingcited by 19
  11. 2022A Visual Analytics System for Improving Attention-based Traffic Forecasting Modelscited by 6
  12. 2018At a Glance: Pixel Approximate Entropy as a Measure of Line Chart Complexitycited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 7; 2026: 5.

18 papers.

Sort:

By area:

  1. 2024Heterogeneity-Informed Meta-Parameter Learning for Spatiotemporal Time Series Forecastingcited by 18
  2. 2024FACTS: A Factored State-Space Framework For World Modellingcited by 4
  3. 2025How Different from the Past? Spatio-Temporal Time Series Forecasting with Self-Supervised Deviation Learningcited by 4
  4. 2024Navigating Spatio-Temporal Heterogeneity: A Graph Transformer Approach for Traffic Forecastingcited by 3
  5. 2025MoGERNN: An Inductive Traffic Predictor for Unobserved Locationscited by 2
  6. 2024Memory-enhanced Invariant Prompt Learning for Urban Flow Prediction under Distribution Shiftscited by 1
  7. 2025ST-LINK: Spatially-Aware Large Language Models for Spatio-Temporal Forecastingcited by 1
  8. 2025HieraMix: A Hierarchical MLP-Mixer for Large-Scale Traffic Forecastingcited by 1
  9. 2024Spatio-Temporal Partial Sensing Forecast for Long-term Trafficcited by 0
  10. 2024HiMoE: Heterogeneity-Informed Mixture-of-Experts for Fair Spatial-Temporal Forecastingcited by 0
  11. 2025DRAN: A Distribution and Relation Adaptive Network for Spatio-temporal Forecastingcited by 0
  12. 2025Spatio-temporal Multivariate Time Series Forecast with Chosen Variablescited by 0
6 more
  1. 2025Towards Resilient Transportation: A Conditional Transformer for Accident-Informed Traffic Forecastingcited by 0
  2. 2026MixTTE: Multi-Level Mixture-of-Experts for Scalable and Adaptive Travel Time Estimationcited by 0
  3. 2026TSNN: A Non-parametric and Interpretable Framework for Traffic Time Series Forecastingcited by 0
  4. 2026Graph-Conditioned Mixture of Graph Neural Network Experts for Traffic Forecastingcited by 0
  5. 2026Frequency-Domain Multi-Modality Transportation Modelingcited by 0
  6. 2026Scene-Conditioned Relation Routing for urban cellular activity forecastingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Less is More: Strategic Expert Selection Outperforms Ensemble Complexity in Traffic Forecasting
  2. 2026GAMMA-Net: Adaptive Long-Horizon Traffic Spatio-Temporal Forecasting Model based on Interleaved Graph Attention and Multi-Axis Mamba
  3. 2026FAST: A Synergistic Framework of Attention and State-space Models for Spatiotemporal Traffic Prediction
  4. 2024Robust Traffic Forecasting against Spatial Shift over Years
  5. 2025DKGCM: A Spatio-Temporal Prediction Model for Traffic Flow by Fusing Spatial Node Clustering Method and Fourier Bidirectional Mamba Mechanism
  6. 2026Spatiotemporal Graph Transformer for Traffic Intelligence in Edge Computing
  7. 2025RAST: A Retrieval Augmented Spatio-Temporal Framework for Traffic Prediction
  8. 2026STHMoE: Hypergraph-Enhanced Heterogeneous Dependency Coordination for LLM-Based Urban Traffic Data Forecasting