mlsanityData to 7 Oct 2026

HiMoE: Heterogeneity-Informed Mixture-of-Experts for Fair Spatial-Temporal Forecasting

arXiv 2412.00316 ↗First posted 30 Nov 2024cs.LG cs.AI cs.CYcited by 0

On the map: Mixture-of-experts → MoE for time series and graphs

Abstract (arXiv)

Achieving both accurate and consistent predictive performance across spatial nodes is crucial for ensuring the validity and reliability of outcomes in fair spatial-temporal forecasting tasks. However, existing training methods treat heterogeneous nodes with a fully averaged perspective, resulting in inherently biased prediction targets. Balancing accuracy and consistency is particularly challenging due to the multi-objective nature of spatial-temporal forecasting. To address this issue, we propose a novel Heterogeneity-Informed Mixture-of-Experts (HiMoE) framework that delivers both uniform and precise spatial-temporal predictions. From a model architecture perspective, we design the Heterogeneity-Informed Graph Convolutional Network (HiGCN) to address trend heterogeneity, and we introduce the Node-wise Mixture-of-Experts (NMoE) module to handle cardinality heterogeneity across nodes. From an evaluation perspective, we propose STFairBench, a benchmark that handles fairness in spatial-temporal prediction from both training and evaluation stages. Extensive experiments on four real-world datasets demonstrate that HiMoE achieves state-of-the-art performance, outperforming the best baseline by at least 9.22% across all evaluation metrics.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 41 of its 58 references that are in the map. First posted on arXiv 30 Nov 2024.

Builds on

41 of 58 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Interpretable Cascading Mixture-of-Experts for Urban Traffic Congestion Prediction14 Jun
  2. 2024TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts5 Mar

Other references · 39

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  6. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,125
  7. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  8. 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 820
  9. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  10. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 563
  11. 2020Adaptive Graph Convolutional Recurrent Network for Traffic Forecastingcited by 525
  12. 2019GMAN: A Graph Multi-Attention Network for Traffic Predictioncited by 433
27 more
  1. 2016Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Predictioncited by 330
  2. 2024A Survey on Mixture of Experts in Large Language Modelscited by 233
  3. 2021Spatial-Temporal Graph ODE Networks for Traffic Flow Forecastingcited by 209
  4. 2024OpenMoE: An Early Effort on Open Mixture-of-Experts Language Modelscited by 185
  5. 2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 179
  6. 2021Efficient Large Scale Language Modeling with Mixtures of Expertscited by 175
  7. 2022Graph Neural Networks for Graphs with Heterophily: A Surveycited by 175
  8. 2023Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Surveycited by 140
  9. 2021Graph Neural Controlled Differential Equations for Traffic Forecastingcited by 133
  10. 2022Decoupled Dynamic Spatial-Temporal Graph Neural Network for Traffic Forecastingcited by 122
  11. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-designcited by 97
  12. 2022Spatio-Temporal Meta-Graph Learning for Traffic Forecastingcited by 88
  13. 2022Spatio-Temporal Self-Supervised Learning for Traffic Flow Predictioncited by 77
  14. 2019Multi-Range Attentive Bicomponent Graph Convolutional Network for Traffic Forecastingcited by 69
  15. 2020PM2.5-GNN: A Domain Knowledge Enhanced Graph Neural Network For PM2.5 Forecastingcited by 52
  16. 2021TrafficStream: A Streaming Traffic Flow Forecasting Framework Based on Graph Neural Networks and Continual Learningcited by 42
  17. 2020Joint Air Quality and Weather Prediction Based on Multi-Adversarial Spatiotemporal Networkscited by 39
  18. 2015Generative Image Modeling Using Spatial LSTMscited by 35
  19. 2023Spatial-Temporal-Decoupled Masked Pre-training for Spatiotemporal Forecastingcited by 27
  20. 2024Heterogeneity-Informed Meta-Parameter Learning for Spatiotemporal Time Series Forecastingcited by 18
  21. 2024FlashST: A Simple and Universal Prompt-Tuning Framework for Traffic Predictioncited by 18
  22. 2023Causal conditional hidden Markov model for multimodal traffic predictioncited by 14
  23. 2019FairST: Equitable Spatial and Temporal Demand Prediction for New Mobility Systemscited by 13
  24. 2023Spatio-temporal neural structural causal models for bike flow predictioncited by 9
  25. 2023Learning Informative Representation for Fairness-aware Multivariate Time-series Forecasting: A Group-based Perspectivecited by 8
  26. 2024FairSTG: Countering performance heterogeneity via collaborative sample-level optimizationcited by 3
  27. 2024Spatial-Temporal Mixture-of-Graph-Experts for Multi-Type Crime Predictioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FaST: Efficient and Effective Long-Horizon Forecasting for Large-Scale Spatial-Temporal Graphs via Mixture-of-Experts
  2. 2022Spatial Mixture-of-Experts
  3. 2026STHMoE: Hypergraph-Enhanced Heterogeneous Dependency Coordination for LLM-Based Urban Traffic Data Forecasting
  4. 2024Navigating Spatio-Temporal Heterogeneity: A Graph Transformer Approach for Traffic Forecasting
  5. 2025Collaborative Deterministic-Probabilistic Forecasting for Diverse Spatiotemporal Systems
  6. 2025Less is More: Strategic Expert Selection Outperforms Ensemble Complexity in Traffic Forecasting
  7. 2023DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Models
  8. 2024ProGen: Revisiting Probabilistic Spatial-Temporal Time Series Forecasting from a Continuous Generative Perspective Using Stochastic Differential Equations