# TrajAgent: An LLM-Agent Framework for Trajectory Modeling via Large-and-Small Model Collaboration

arXiv: https://arxiv.org/abs/2410.20445 · HTML: https://mlsanity.com/paper/2410.20445/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 20 of its 72 references that are in the map.
- First posted on arXiv 27 Oct 2024.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 20 of 20)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks](https://mlsanity.com/paper/2406.08394/) (2024)
- [SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories](https://mlsanity.com/paper/2409.07440/) (2024)
- [Benchmarking Data Science Agents](https://mlsanity.com/paper/2402.17168/) (2024)
- [Trajectory Data Management and Mining: A Survey from Deep Learning to the LLM Era](https://mlsanity.com/paper/2403.14151/) (2024)
- [Large Language Models are Zero-Shot Next Location Predictors](https://mlsanity.com/paper/2405.20962/) (2024)
- [TrajFM: A Vehicle Trajectory Foundation Model for Region and Task Transferability](https://mlsanity.com/paper/2408.15251/) (2024)
- [Mutual Distillation Learning Network for Trajectory-User Linking](https://mlsanity.com/paper/2205.03773/) (2022)
- [UrbanLLM: Autonomous Urban Activity Planning and Management with Large Language Models](https://mlsanity.com/paper/2406.12360/) (2024)
- [TrajectoryNet: An Embedded GPS Trajectory Representation for Point-based Classification Using Recurrent Neural Networks](https://mlsanity.com/paper/1705.02636/) (2017)
- [Pretrained Mobility Transformer: A Foundation Model for Human Mobility](https://mlsanity.com/paper/2406.02578/) (2024)
- [LIMP: Large Language Model Enhanced Intent-aware Mobility Prediction](https://mlsanity.com/paper/2408.12832/) (2024)

## Built on by (most cited first, 5 of 5)

- [CAMS: A CityGPT-Powered Agentic Framework for Urban Human Mobility Simulation](https://mlsanity.com/paper/2506.13599/) (2025)
- [A Benchmark for Procedural Memory Retrieval in Language Agents](https://mlsanity.com/paper/2511.21730/) (2025)
- [Adaptive Location Hierarchy Learning for Long-Tailed Mobility Prediction](https://mlsanity.com/paper/2505.19965/) (2025)
- [AutoQual: An LLM Agent for Automated Discovery of Interpretable Features for Review Quality Assessment](https://mlsanity.com/paper/2510.08081/) (2025)
- [ARMove: Learning to Predict Human Mobility through Agentic Reasoning](https://mlsanity.com/paper/2604.17419/) (2026)

## Abstract (arXiv)

Trajectory modeling, which includes research on trajectory data pattern mining and future prediction, has widespread applications in areas such as life services, urban transportation, and public administration. Numerous methods have been proposed to address specific problems within trajectory modeling. However, the heterogeneity of data and the diversity of trajectory tasks make effective and reliable trajectory modeling an important yet highly challenging endeavor, even for domain experts. In this paper, we propose TrajAgent, an agent framework powered by large language models, designed to facilitate robust and efficient trajectory modeling through automation modeling. This framework leverages and optimizes diverse specialized models to address various trajectory modeling tasks across different datasets effectively. In TrajAgent, we first develop UniEnv, an execution environment with a unified data and model interface, to support the execution and training of various models. Building on UniEnv, we introduce an agentic workflow designed for automatic trajectory modeling across various trajectory tasks and data. Furthermore, we introduce collaborative learning schema between LLM-based agents and small speciallized models, to enhance the performance of the whole framework effectively. Extensive experiments on five tasks using four real-world datasets demonstrate the effectiveness of TrajAgent in automated trajectory modeling, achieving a performance improvement of 2.38%-69.91% over baseline methods. The codes and data can be accessed via https://github.com/tsinghua-fib-lab/TrajAgent.