# Learning Generalized and Flexible Trajectory Models from Omni-Semantic Supervision

arXiv: https://arxiv.org/abs/2505.17437 · HTML: https://mlsanity.com/paper/2505.17437/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 46 references that are in the map.
- First posted on arXiv 23 May 2025.

## Builds on (most cited first, 15 of 15)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [ControlTraj: Controllable Trajectory Generation with Topology-Constrained Diffusion Model](https://mlsanity.com/paper/2404.15380/) (2024)
- [Contrastive Trajectory Similarity Learning with Dual-Feature Attention](https://mlsanity.com/paper/2210.05155/) (2022)
- [A Survey on Trajectory Data Management, Analytics, and Learning](https://mlsanity.com/paper/2003.11547/) (2020)
- [Deep Spatially and Temporally Aware Similarity Computation for Road Network Constrained Trajectories](https://mlsanity.com/paper/2112.09339/) (2021)
- [Pre-training General Trajectory Embeddings with Maximum Multi-view Entropy Coding](https://mlsanity.com/paper/2207.14539/) (2022)
- [A Multi-view Multi-task Learning Framework for Multi-variate Time Series Forecasting](https://mlsanity.com/paper/2109.01657/) (2021)
- [Trajectory Similarity Measurement: An Efficiency Perspective](https://mlsanity.com/paper/2311.00960/) (2023)
- [TrajFM: A Vehicle Trajectory Foundation Model for Region and Task Transferability](https://mlsanity.com/paper/2408.15251/) (2024)
- [MM-Path: Multi-modal, Multi-granularity Path Representation Learning -- Extended Version](https://mlsanity.com/paper/2411.18428/) (2024)
- [Grid and Road Expressions Are Complementary for Trajectory Representation Learning](https://mlsanity.com/paper/2411.14768/) (2024)

## Built on by (most cited first, 1 of 1)

- [Multimodal Trajectory Representation Learning for Travel Time Estimation](https://mlsanity.com/paper/2510.05840/) (2025)

## Abstract (arXiv)

The widespread adoption of mobile devices and data collection technologies has led to an exponential increase in trajectory data, presenting significant challenges in spatio-temporal data mining, particularly for efficient and accurate trajectory retrieval. However, existing methods for trajectory retrieval face notable limitations, including inefficiencies in large-scale data, lack of support for condition-based queries, and reliance on trajectory similarity measures. To address the above challenges, we propose OmniTraj, a generalized and flexible omni-semantic trajectory retrieval framework that integrates four complementary modalities or semantics -- raw trajectories, topology, road segments, and regions -- into a unified system. Unlike traditional approaches that are limited to computing and processing trajectories as a single modality, OmniTraj designs dedicated encoders for each modality, which are embedded and fused into a shared representation space. This design enables OmniTraj to support accurate and flexible queries based on any individual modality or combination thereof, overcoming the rigidity of traditional similarity-based methods. Extensive experiments on two real-world datasets demonstrate the effectiveness of OmniTraj in handling large-scale data, providing flexible, multi-modality queries, and supporting downstream tasks and applications.