Building a Foundation Model for Trajectory from Scratch
Not on the map. Why?
Abstract (arXiv)
Foundation models are transformative in artificial intelligence, but building them from scratch, especially for mobility trajectories, is not yet clear or documented. This tutorial bridges this gap by demonstrating the steps and code of a minimal implementation of a trajectory-focused foundation model starting from GPT-2. Through a concise, step-by-step, code-driven process, we demonstrate adapting GPT-2 for spatiotemporal data. We then review and compare representative trajectory foundation models, such as TrajFM and TrajGPT, highlighting their architectural innovations and differences. Additionally, we introduce complementary techniques from related domains, like TimesFM's patching approach. Targeted at researchers and practitioners, this tutorial aims to explain the concepts and terminology of foundation models, at the implementation level. We find it timely and indispensable to create this educational material in order to support the SIGSPATIAL community in building and evaluating mobility foundation models, enhancing both research clarity and peer-review effectiveness in mobility AI.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 14 references that are in the map. First posted on arXiv 25 Nov 2025.
Builds on
6 of 14 references.
References · 6
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
- 2019Time2Vec: Learning a Vector Representation of Timecited by 213
- 2020Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cellscited by 90
- 2024TrajFM: A Vehicle Trajectory Foundation Model for Region and Task Transferabilitycited by 16
- 2024TrajGPT: Controlled Synthetic Trajectory Generation Using a Multitask Transformer-Based Spatiotemporal Modelcited by 13
Built on by
Citing papers per year: 2026: 1.
1 paper.