# TrajTok: Adaptive Spatial Tokenization for Trajectory Representation Learning

arXiv: https://arxiv.org/abs/2605.20134 · HTML: https://mlsanity.com/paper/2605.20134/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 19 May 2026.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery](https://mlsanity.com/paper/2311.17179/) (2023)
- [Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells](https://mlsanity.com/paper/2003.00824/) (2020)
- [Self-supervised Trajectory Representation Learning with Temporal Regularities and Travel Semantics](https://mlsanity.com/paper/2211.09510/) (2022)
- [Contrastive Trajectory Similarity Learning with Dual-Feature Attention](https://mlsanity.com/paper/2210.05155/) (2022)
- [Unsupervised Path Representation Learning with Curriculum Negative Sampling](https://mlsanity.com/paper/2106.09373/) (2021)
- [Pre-training General Trajectory Embeddings with Maximum Multi-view Entropy Coding](https://mlsanity.com/paper/2207.14539/) (2022)
- [More Than Routing: Joint GPS and Route Modeling for Refine Trajectory Representation Learning](https://mlsanity.com/paper/2402.16915/) (2024)
- [SIMformer: Single-Layer Vanilla Transformer Can Learn Free-Space Trajectory Similarity](https://mlsanity.com/paper/2410.14629/) (2024)
- [The Fine Line: Navigating Large Language Model Pretraining with Down-streaming Capability Analysis](https://mlsanity.com/paper/2404.01204/) (2024)
- [Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima](https://mlsanity.com/paper/2604.09258/) (2026)
- [Towards A Foundation Model For Trajectory Intelligence](https://mlsanity.com/paper/2312.00076/) (2023)
- [Trajectory Representation Learning on Road Networks and Grids with Spatio-Temporal Dynamics](https://mlsanity.com/paper/2411.14014/) (2024)
- [Region-Point Joint Representation for Effective Trajectory Similarity Learning](https://mlsanity.com/paper/2511.13125/) (2025)

## Built on by (most cited first, 1 of 1)

- [Industrial Kinematic Trajectory Model (IKTM): Coordinate-Free Autoregressive Generator](https://mlsanity.com/paper/2609.22173/) (2026)

## Abstract (arXiv)

Learning generalizable trajectory representations from raw GPS traces remains difficult because the data is continuous, noisy, and irregularly sampled. Spatial tokenization is also challenging: fine grids yield sparse cells with weak embeddings, while coarse grids merge heterogeneous movement patterns into the same token. We present TrajTok, a trajectory encoder with a simple pretraining recipe for transferable trajectory embeddings. TrajTok first learns a multi-resolution hexagonal cell partition from the spatial distribution of GPS points, converting noisy GPS sequences into discrete cell tokens. To capture both geometry and kinematics, it uses a factorized transformer encoder with early per-modality self-attention blocks, cross-attention fusion layers, and spatiotemporal rotary position embeddings, ST-RoPE, to encode where and when each token occurs. TrajTok is pretrained with masked-token modeling that recovers both geometric structure and kinematic patterns from partial trajectory observations. On the Porto dataset, a frozen TrajTok encoder with lightweight task adapters achieves strong performance across trajectory similarity search, classification, estimated time of arrival, and full travel-time regression, outperforming multiple task-specific methods. The same frozen encoder supports both geometry-dominated and kinematics-dominated tasks, suggesting that TrajTok learns transferable trajectory structure rather than task-specific shortcuts. These results indicate that learned multi-resolution spatial tokenization combined with masked-token pretraining is a promising direction for general-purpose trajectory foundation models.