mlsanityData to 7 Oct 2026

ST-MoE-BERT: A Spatial-Temporal Mixture-of-Experts Framework for Long-Term Cross-City Mobility Prediction

arXiv 2410.14099 ↗First posted 18 Oct 2024cs.LG cs.AIcited by 7

On the map: Mixture-of-experts → MoE for time series and graphs

Abstract (arXiv)

Predicting human mobility across multiple cities presents significant challenges due to the complex and diverse spatial-temporal dynamics inherent in different urban environments. In this study, we propose a robust approach to predict human mobility patterns called ST-MoE-BERT. Compared to existing methods, our approach frames the prediction task as a spatial-temporal classification problem. Our methodology integrates the Mixture-of-Experts architecture with BERT model to capture complex mobility dynamics and perform the downstream human mobility prediction task. Additionally, transfer learning is integrated to solve the challenge of data scarcity in cross-city prediction. We demonstrate the effectiveness of the proposed model on GEO-BLEU and DTW, comparing it to several state-of-the-art methods. Notably, ST-MoE-BERT achieves an average improvement of 8.29%.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 15 of its 42 references that are in the map. First posted on arXiv 18 Oct 2024.

Builds on

15 of 42 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  3. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  4. 2023BloombergGPT: A Large Language Model for Financecited by 923
  5. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 187
  6. 2021CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPcited by 88
  7. 2020A Survey on Deep Learning for Human Mobilitycited by 87
  8. 2023FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasetscited by 68
  9. 2021MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Predictioncited by 48
  10. 2023Exploring Large Language Models for Human Mobility Prediction under Public Eventscited by 34
  11. 2024COLA: Cross-city Mobility Transformer for Human Trajectory Simulationcited by 25
  12. 2024Large Language Models are Zero-Shot Next Location Predictorscited by 22
3 more
  1. 2024Decoupled Alignment for Robust Plug-and-Play Adaptationcited by 9
  2. 2021GEO-BLEU: Similarity Measure for Geospatial Sequencescited by 7
  3. 2024Mixing Individual and Collective Behaviours to Predict Out-of-Routine Mobilitycited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2026FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoningcited by 5
  2. 2025UniMove: A Unified Model for Multi-city Human Mobility Predictioncited by 2
  3. 2026Attention Sinks and Outliers in Attention Residualscited by 2
  4. 2025Efficient Temporal Tokenization for Mobility Prediction with Large Language Modelscited by 0
  5. 2025User Trajectory Prediction Unifying Global and Local Temporal Informationcited by 0
  6. 2025RAST-MoE-RL: A Regime-Aware Spatio-Temporal MoE Framework for Deep Reinforcement Learning in Ride-Hailingcited by 0
  7. 2026Evolutionary Router Feature Generation for Zero-Shot Graph Anomaly Detection with Mixture-of-Expertscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Mixture-of-Experts for Personalized and Semantic-Aware Next Location Prediction
  2. 2025MoE-TransMov: A Transformer-based Model for Next POI Prediction in Familiar & Unfamiliar Movements
  3. 2025MoveGPT: Scaling Mobility Foundation Models with Spatially-Aware Mixture of Experts
  4. 2025Adaptive Location Hierarchy Learning for Long-Tailed Mobility Prediction
  5. 2025SeMob: Semantic Synthesis for Dynamic Urban Mobility Prediction
  6. 2024A Universal Model for Human Mobility Prediction
  7. 2023Where Would I Go Next? Large Language Models as Human Mobility Predictors
  8. 2026ARMove: Learning to Predict Human Mobility through Agentic Reasoning