# Classical Feature Embeddings Help in BERT-Based Human Mobility Prediction

arXiv: https://arxiv.org/abs/2510.20275 · HTML: https://mlsanity.com/paper/2510.20275/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 23 Oct 2025.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GEO-BLEU: Similarity Measure for Geospatial Sequences](https://mlsanity.com/paper/2112.07144/) (2021)
- [Llama-Mob: Instruction-Tuning Llama-3-8B Excels in City-Scale Mobility Prediction](https://mlsanity.com/paper/2410.23692/) (2024)
- [GeoFormer: Predicting Human Mobility using Generative Pre-trained Transformer (GPT)](https://mlsanity.com/paper/2311.05092/) (2023)
- [Personalized human mobility prediction for HuMob challenge](https://mlsanity.com/paper/2310.12900/) (2023)

## Abstract (arXiv)

Human mobility forecasting is crucial for disaster relief, city planning, and public health. However, existing models either only model location sequences or include time information merely as auxiliary input, thereby failing to leverage the rich semantic context provided by points of interest (POIs). To address this, we enrich a BERT-based mobility model with derived temporal descriptors and POI embeddings to better capture the semantics underlying human movement. We propose STaBERT (Semantic-Temporal aware BERT), which integrates both POI and temporal information at each location to construct a unified, semantically enriched representation of mobility. Experimental results show that STaBERT significantly improves prediction accuracy: for single-city prediction, the GEO-BLEU score improved from 0.34 to 0.75; for multi-city prediction, from 0.34 to 0.56.