# TrajGPT: Controlled Synthetic Trajectory Generation Using a Multitask Transformer-Based Spatiotemporal Model

arXiv: https://arxiv.org/abs/2411.04381 · HTML: https://mlsanity.com/paper/2411.04381/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 7 Nov 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [Time2Vec: Learning a Vector Representation of Time](https://mlsanity.com/paper/1907.05321/) (2019)
- [Enabling Language Models to Fill in the Blanks](https://mlsanity.com/paper/2005.05339/) (2020)
- [STAN: Spatio-Temporal Attention Network for Next Location Recommendation](https://mlsanity.com/paper/2102.04095/) (2021)
- [Where Would I Go Next? Large Language Models as Human Mobility Predictors](https://mlsanity.com/paper/2308.15197/) (2023)
- [Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells](https://mlsanity.com/paper/2003.00824/) (2020)
- [GETNext: Trajectory Flow Map Enhanced Transformer for Next POI Recommendation](https://mlsanity.com/paper/2303.04741/) (2023)
- [MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Prediction](https://mlsanity.com/paper/2110.01401/) (2021)
- [Blank Language Models](https://mlsanity.com/paper/2002.03079/) (2020)
- [A Hierarchical Spatial Transformer for Massive Point Samples in Continuous Space](https://mlsanity.com/paper/2311.04434/) (2023)
- [Hierarchical Transformer with Spatio-Temporal Context Aggregation for Next Point-of-Interest Recommendation](https://mlsanity.com/paper/2209.01559/) (2022)

## Built on by (most cited first, 13 of 13)

- [TrajFlow: Nation-wide Pseudo GPS Trajectory Generation with Flow Matching Models](https://mlsanity.com/paper/2603.15009/) (2026)
- [Mobility-Embedded POIs: Learning What A Place Is and How It Is Used from Human Movement](https://mlsanity.com/paper/2601.21149/) (2026)
- [Forecasting Unseen Points of Interest Visits Using Context and Proximity Priors](https://mlsanity.com/paper/2411.15285/) (2024)
- [MoveGPT: Scaling Mobility Foundation Models with Spatially-Aware Mixture of Experts](https://mlsanity.com/paper/2505.18670/) (2025)
- [SVBRD-LLM: Self-Verifying Behavioral Rule Discovery for Autonomous Vehicle Identification](https://mlsanity.com/paper/2511.14977/) (2025)
- [Building a Foundation Model for Trajectory from Scratch](https://mlsanity.com/paper/2511.20610/) (2025)
- [POIFormer: A Transformer-Based Framework for Accurate and Scalable Point-of-Interest Attribution](https://mlsanity.com/paper/2507.09137/) (2025)
- [Efficient Temporal Tokenization for Mobility Prediction with Large Language Models](https://mlsanity.com/paper/2507.14017/) (2025)
- [GPS-MTM: Capturing Pattern of Normalcy in GPS-Trajectories with self-supervised learning](https://mlsanity.com/paper/2509.24031/) (2025)
- [TrajGPT-R: Generating Urban Mobility Trajectory with Reinforcement Learning-Enhanced Generative Pre-trained Transformer](https://mlsanity.com/paper/2602.20643/) (2026)
- [Learning Demographic-Conditioned Mobility Trajectories with Aggregate Supervision](https://mlsanity.com/paper/2603.03275/) (2026)
- [TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond](https://mlsanity.com/paper/2605.06906/) (2026)
- [diffGHOST: Diffusion based Generative Hedged Oblivious Synthetic Trajectories](https://mlsanity.com/paper/2605.10647/) (2026)

## Abstract (arXiv)

Human mobility modeling from GPS-trajectories and synthetic trajectory generation are crucial for various applications, such as urban planning, disaster management and epidemiology. Both of these tasks often require filling gaps in a partially specified sequence of visits - a new problem that we call "controlled" synthetic trajectory generation. Existing methods for next-location prediction or synthetic trajectory generation cannot solve this problem as they lack the mechanisms needed to constrain the generated sequences of visits. Moreover, existing approaches (1) frequently treat space and time as independent factors, an assumption that fails to hold true in real-world scenarios, and (2) suffer from challenges in accuracy of temporal prediction as they fail to deal with mixed distributions and the inter-relationships of different modes with latent variables (e.g., day-of-the-week). These limitations become even more pronounced when the task involves filling gaps within sequences instead of solely predicting the next visit. We introduce TrajGPT, a transformer-based, multi-task, joint spatiotemporal generative model to address these issues. Taking inspiration from large language models, TrajGPT poses the problem of controlled trajectory generation as that of text infilling in natural language. TrajGPT integrates the spatial and temporal models in a transformer architecture through a Bayesian probability model that ensures that the gaps in a visit sequence are filled in a spatiotemporally consistent manner. Our experiments on public and private datasets demonstrate that TrajGPT not only excels in controlled synthetic visit generation but also outperforms competing models in next-location prediction tasks - Relatively, TrajGPT achieves a 26-fold improvement in temporal accuracy while retaining more than 98% of spatial accuracy on average.