# Pretrained Mobility Transformer: A Foundation Model for Human Mobility

arXiv: https://arxiv.org/abs/2406.02578 · HTML: https://mlsanity.com/paper/2406.02578/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 69 references that are in the map.
- First posted on arXiv 29 May 2024.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Revisiting Spatial-Temporal Similarity: A Deep Learning Framework for Traffic Prediction](https://mlsanity.com/paper/1803.01254/) (2018)
- [Predicting Citywide Crowd Flows in Irregular Regions Using Multi-View Graph Convolutional Networks](https://mlsanity.com/paper/1903.07789/) (2019)
- [Learning Geo-Contextual Embeddings for Commuting Flow Prediction](https://mlsanity.com/paper/2005.01690/) (2020)
- [COLA: Cross-city Mobility Transformer for Human Trajectory Simulation](https://mlsanity.com/paper/2403.01801/) (2024)
- [Learning to Simulate Daily Activities via Modeling Dynamic Human Needs](https://mlsanity.com/paper/2302.10897/) (2023)
- [Generating synthetic mobility data for a realistic population with RNNs to improve utility and privacy](https://mlsanity.com/paper/2201.01139/) (2022)
- [The Effect of Recency to Human Mobility](https://mlsanity.com/paper/1504.01442/) (2015)
- [Modelling Human Mobility considering Spatial,Temporal and Social Dimensions](https://mlsanity.com/paper/2007.02371/) (2020)
- [Social media and mobility landscape: uncovering spatial patterns of urban human mobility with multi source data](https://mlsanity.com/paper/1808.04422/) (2018)

## Built on by (most cited first, 8 of 8)

- [TrajAgent: An LLM-Agent Framework for Trajectory Modeling via Large-and-Small Model Collaboration](https://mlsanity.com/paper/2410.20445/) (2024)
- [RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility](https://mlsanity.com/paper/2509.23115/) (2025)
- [From Points to Places: Towards Human Mobility-Driven Spatiotemporal Foundation Models via Understanding Places](https://mlsanity.com/paper/2506.14570/) (2025)
- [PlaceRep: Geospatial Place Representation Learning from Large-Scale Point-of-Interest Data](https://mlsanity.com/paper/2507.02921/) (2025)
- [MoveFM-R: Advancing Mobility Foundation Models via Language-driven Semantic Reasoning](https://mlsanity.com/paper/2509.22403/) (2025)
- [Efficient Temporal Tokenization for Mobility Prediction with Large Language Models](https://mlsanity.com/paper/2507.14017/) (2025)
- [On Predicting Sociodemographics from Mobility Signals](https://mlsanity.com/paper/2511.03924/) (2025)
- [Coarse-to-Fine Multi-Resolution Diffusion Models for Trajectory Generation in Urban Systems](https://mlsanity.com/paper/2608.14570/) (2026)

## Abstract (arXiv)

Ubiquitous mobile devices are generating vast amounts of location-based service data that reveal how individuals navigate and utilize urban spaces in detail. In this study, we utilize these extensive, unlabeled sequences of user trajectories to develop a foundation model for understanding urban space and human mobility. We introduce the Pretrained Mobility Transformer (PMT), which leverages the transformer architecture to process user trajectories in an autoregressive manner, converting geographical areas into tokens and embedding spatial and temporal information within these representations. Experiments conducted in three U.S. metropolitan areas over a two-month period demonstrate PMT's ability to capture underlying geographic and socio-demographic characteristics of regions. The proposed PMT excels across various downstream tasks, including next-location prediction, trajectory imputation, and trajectory generation. These results support PMT's capability and effectiveness in decoding complex patterns of human mobility, offering new insights into urban spatial functionality and individual mobility preferences.