# UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces

arXiv: https://arxiv.org/abs/2411.03859 · HTML: https://mlsanity.com/paper/2411.03859/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 55 references that are in the map.
- First posted on arXiv 6 Nov 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [A Survey on Deep Learning for Human Mobility](https://mlsanity.com/paper/2012.02825/) (2020)
- [On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligence](https://mlsanity.com/paper/2304.06798/) (2023)
- [UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction](https://mlsanity.com/paper/2402.11838/) (2024)
- [Self-supervised Trajectory Representation Learning with Temporal Regularities and Travel Semantics](https://mlsanity.com/paper/2211.09510/) (2022)
- [ControlTraj: Controllable Trajectory Generation with Topology-Constrained Diffusion Model](https://mlsanity.com/paper/2404.15380/) (2024)
- [A Survey on Trajectory Data Management, Analytics, and Learning](https://mlsanity.com/paper/2003.11547/) (2020)
- [OpenCity: Open Spatio-Temporal Foundation Models for Traffic Prediction](https://mlsanity.com/paper/2408.10269/) (2024)
- [Inferring transportation modes from GPS trajectories using a convolutional neural network](https://mlsanity.com/paper/1804.02386/) (2018)
- [Pre-training General Trajectory Embeddings with Maximum Multi-view Entropy Coding](https://mlsanity.com/paper/2207.14539/) (2022)
- [More Than Routing: Joint GPS and Route Modeling for Refine Trajectory Representation Learning](https://mlsanity.com/paper/2402.16915/) (2024)
- [LightTR: A Lightweight Framework for Federated Trajectory Recovery](https://mlsanity.com/paper/2405.03409/) (2024)
- [Trajectory Similarity Measurement: An Efficiency Perspective](https://mlsanity.com/paper/2311.00960/) (2023)
- [TrajFM: A Vehicle Trajectory Foundation Model for Region and Task Transferability](https://mlsanity.com/paper/2408.15251/) (2024)
- [Learning to Route with Sparse Trajectory Sets---Extended Version](https://mlsanity.com/paper/1802.07980/) (2018)
- [Experience: Understanding Long-Term Evolving Patterns of Shared Electric Vehicle Networks](https://mlsanity.com/paper/1812.07499/) (2018)

## Built on by (most cited first, 18 of 18)

- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [HiT-JEPA: A Hierarchical Self-supervised Trajectory Embedding Framework for Similarity Computation](https://mlsanity.com/paper/2507.00028/) (2025)
- [GeoArena: Evaluating Open-World Geographic Reasoning in Large Vision-Language Models](https://mlsanity.com/paper/2509.04334/) (2025)
- [From Points to Places: Towards Human Mobility-Driven Spatiotemporal Foundation Models via Understanding Places](https://mlsanity.com/paper/2506.14570/) (2025)
- [Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Review](https://mlsanity.com/paper/2506.14831/) (2025)
- [TrajDLM: Topology-Aware Block Diffusion Language Model for Trajectory Generation](https://mlsanity.com/paper/2605.10020/) (2026)
- [Text-to-TrajVis: Enabling Trajectory Data Visualizations from Natural Language Questions](https://mlsanity.com/paper/2504.16358/) (2025)
- [MoveFM-R: Advancing Mobility Foundation Models via Language-driven Semantic Reasoning](https://mlsanity.com/paper/2509.22403/) (2025)
- [Understanding the Geospatial Reasoning Capabilities of LLMs: A Trajectory Recovery Perspective](https://mlsanity.com/paper/2510.01639/) (2025)
- [Embedding-Aware Feature Discovery: Bridging Latent Representations and Interpretable Features in Event Sequences](https://mlsanity.com/paper/2603.15713/) (2026)
- [TrajPrism: A Multi-Task Benchmark for Language-Grounded Urban Trajectory Understanding](https://mlsanity.com/paper/2605.10782/) (2026)
- [Hyperspectral Trajectory Image for Multi-Month Trajectory Anomaly Detection](https://mlsanity.com/paper/2603.25255/) (2026)
- [TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond](https://mlsanity.com/paper/2605.06906/) (2026)
- [TrajGANR: Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations](https://mlsanity.com/paper/2605.06990/) (2026)
- [Mobility Anomaly Generation using LLM-Driven Behavior with Kinematic Constraints](https://mlsanity.com/paper/2606.10314/) (2026)
- [TrajMind: Chaining Role-Specialized LoRAs for Fast-and-Slow Collective Trajectory Anomaly Diagnosis](https://mlsanity.com/paper/2609.02540/) (2026)
- [SSP-DMGTimeNet: Physics-Constrained Learning for Spatiotemporal Trajectory Prediction of Vehicle Platoons](https://mlsanity.com/paper/2609.06961/) (2026)
- [Industrial Kinematic Trajectory Model (IKTM): Coordinate-Free Autoregressive Generator](https://mlsanity.com/paper/2609.22173/) (2026)

## Abstract (arXiv)

Building a universal trajectory foundation model is a promising solution to address the limitations of existing trajectory modeling approaches, such as task specificity, regional dependency, and data sensitivity. Despite its potential, data preparation, pre-training strategy development, and architectural design present significant challenges in constructing this model. Therefore, we introduce UniTraj, a Universal Trajectory foundation model that aims to address these limitations through three key innovations. First, we construct WorldTrace, an unprecedented dataset of 2.45 million trajectories with billions of GPS points spanning 70 countries, providing the diverse geographic coverage essential for region-independent modeling. Second, we develop novel pre-training strategies--Adaptive Trajectory Resampling and Self-supervised Trajectory Masking--that enable robust learning from heterogeneous trajectory data with varying sampling rates and quality. Finally, we tailor a flexible model architecture to accommodate a variety of trajectory tasks, effectively capturing complex movement patterns to support broad applicability. Extensive experiments across multiple tasks and real-world datasets demonstrate that UniTraj consistently outperforms existing methods, exhibiting superior scalability, adaptability, and generalization, with WorldTrace serving as an ideal yet non-exclusive training resource.