# TrajPrism: A Multi-Task Benchmark for Language-Grounded Urban Trajectory Understanding

arXiv: https://arxiv.org/abs/2605.10782 · HTML: https://mlsanity.com/paper/2605.10782/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 11 May 2026.

## Builds on (most cited first, 26 of 26)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [TravelPlanner: A Benchmark for Real-World Planning with Language Agents](https://mlsanity.com/paper/2402.01622/) (2024)
- [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://mlsanity.com/paper/2402.01613/) (2024)
- [Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generation](https://mlsanity.com/paper/2402.14744/) (2024)
- [UrbanGPT: Spatio-Temporal Large Language Models](https://mlsanity.com/paper/2403.00813/) (2024)
- [Contrastive Trajectory Similarity Learning with Dual-Feature Attention](https://mlsanity.com/paper/2210.05155/) (2022)
- [Mobility-LLM: Learning Visiting Intentions and Travel Preferences from Human Mobility Data with Large Language Models](https://mlsanity.com/paper/2411.00823/) (2024)
- [More Than Routing: Joint GPS and Route Modeling for Refine Trajectory Representation Learning](https://mlsanity.com/paper/2402.16915/) (2024)
- [UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces](https://mlsanity.com/paper/2411.03859/) (2024)
- [T-JEPA: A Joint-Embedding Predictive Architecture for Trajectory Similarity Computation](https://mlsanity.com/paper/2406.12913/) (2024)
- [Holistic Semantic Representation for Navigational Trajectory Generation](https://mlsanity.com/paper/2501.02737/) (2025)
- [Blurred Encoding for Trajectory Representation Learning](https://mlsanity.com/paper/2511.13741/) (2025)
- [MobilityBench: A Benchmark for Evaluating Route-Planning Agents in Real-World Mobility Scenarios](https://mlsanity.com/paper/2602.22638/) (2026)
- [GenUP: Generative User Profilers as In-Context Learners for Next POI Recommender Systems](https://mlsanity.com/paper/2410.20643/) (2024)
- [HiT-JEPA: A Hierarchical Self-supervised Trajectory Embedding Framework for Similarity Computation](https://mlsanity.com/paper/2507.00028/) (2025)
- [CausalMob: Causal Human Mobility Prediction with LLMs-derived Human Intentions toward Public Events](https://mlsanity.com/paper/2412.02155/) (2024)
- [UniTE: A Survey and Unified Pipeline for Pre-training Spatiotemporal Trajectory Embeddings](https://mlsanity.com/paper/2407.12550/) (2024)
- [Large Language Model Powered Intelligent Urban Agents: Concepts, Capabilities, and Applications](https://mlsanity.com/paper/2507.00914/) (2025)
- [ELLMob: Event-Driven Human Mobility Generation with Self-Aligned LLM Framework](https://mlsanity.com/paper/2603.07946/) (2026)
- [Massive-STEPS: Massive Semantic Trajectories for Understanding POI Check-ins -- Dataset and Benchmarks](https://mlsanity.com/paper/2505.11239/) (2025)

## Built on by (most cited first, 1 of 1)

- [Driving, Fast or Slow? Neuro-Symbolic Guidance for Motion Prediction in Multi-Modal Ground Mobility](https://mlsanity.com/paper/2606.15251/) (2026)

## Abstract (arXiv)

Urban mobility is naturally expressed both as trajectories in space and as natural-language descriptions of travel intent, constraints, and preferences. However, prior work rarely evaluates these two modalities together on the same real-world trajectories: trajectory modeling often stays geometry-centric, while language-centric mobility benchmarks frequently target route planning and tool use rather than fine-grained, verifiable alignment between text and the underlying route. We introduce TrajPrism, a multi-task benchmark for language-trajectory alignment that unifies (i) instruction-conditioned trajectory generation, (ii) language-driven semantic trajectory retrieval, and (iii) trajectory captioning, together with an evaluation protocol that measures trajectory fidelity, retrieval quality, and language groundedness. We construct TrajPrism by pairing real urban trajectories with judge-filtered language annotations generated under a four-dimensional travel-intent taxonomy. The benchmark contains 300K selected trajectories across Porto, San Francisco, and Beijing, yielding 2.1M task instances from three instruction variants, three retrieval queries, and one caption per trajectory. We further develop proof-of-concept models for each task: TrajAnchor for instruction-conditioned trajectory generation, TrajFuse for semantic trajectory retrieval, and TrajRap for trajectory captioning. These models instantiate the proposed tasks and show that geometry-only trajectory baselines leave a large gap on our protocol, especially where language is part of the input-output interface. We release TrajPrism with code and a reproducible annotation pipeline that is designed to be portable across cities, given compatible trajectory inputs and map resources.