# PLMTrajRec: A Scalable and Generalizable Trajectory Recovery Method with Pre-trained Language Models

arXiv: https://arxiv.org/abs/2410.14281 · HTML: https://mlsanity.com/paper/2410.14281/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 53 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024)
- [VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks](https://mlsanity.com/paper/2305.11175/) (2023)
- [TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting](https://mlsanity.com/paper/2310.04948/) (2023)
- [TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series](https://mlsanity.com/paper/2308.08241/) (2023)
- [Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding](https://mlsanity.com/paper/2106.02795/) (2021)
- [GenRec: Large Language Model for Generative Recommendation](https://mlsanity.com/paper/2307.00457/) (2023)
- [A Survey on Trajectory Data Management, Analytics, and Learning](https://mlsanity.com/paper/2003.11547/) (2020)
- [Diff-RNTraj: A Structure-aware Diffusion Model for Road Network-constrained Trajectory Generation](https://mlsanity.com/paper/2402.07369/) (2024)
- [COLA: Cross-city Mobility Transformer for Human Trajectory Simulation](https://mlsanity.com/paper/2403.01801/) (2024)
- [RNTrajRec: Road Network Enhanced Trajectory Recovery with Spatial-Temporal Transformer](https://mlsanity.com/paper/2211.13234/) (2022)
- [City Foundation Models for Learning General Purpose Representations from OpenStreetMap](https://mlsanity.com/paper/2310.00583/) (2023)
- [LightTR: A Lightweight Framework for Federated Trajectory Recovery](https://mlsanity.com/paper/2405.03409/) (2024)
- [Trajectory Recovery From Ash: User Privacy Is NOT Preserved in Aggregated Mobility Data](https://mlsanity.com/paper/1702.06270/) (2017)
- [Learning Points and Routes to Recommend Trajectories](https://mlsanity.com/paper/1608.07051/) (2016)
- [AttnMove: History Enhanced Trajectory Recovery via Attentional Network](https://mlsanity.com/paper/2101.00646/) (2021)
- [Micro-Macro Spatial-Temporal Graph-based Encoder-Decoder for Map-Constrained Trajectory Recovery](https://mlsanity.com/paper/2404.19141/) (2024)
- [Inferring Uncertain Trajectories from Partial Observations](https://mlsanity.com/paper/1603.07641/) (2016)
- [STMGF: An Effective Spatial-Temporal Multi-Granularity Framework for Traffic Forecasting](https://mlsanity.com/paper/2404.05774/) (2024)

## Abstract (arXiv)

Spatiotemporal trajectory data is crucial for various applications. However, issues such as device malfunctions and network instability often cause sparse trajectories, leading to lost detailed movement information. Recovering the missing points in sparse trajectories to restore the detailed information is thus essential. Despite recent progress, several challenges remain. First, the lack of large-scale dense trajectory data makes it difficult to train a trajectory recovery model from scratch. Second, the varying spatiotemporal correlations in sparse trajectories make it hard to generalize recovery across different sampling intervals. Third, the lack of location information complicates the extraction of road conditions for missing points. To address these challenges, we propose a novel trajectory recovery model called PLMTrajRec. It leverages the scalability of a pre-trained language model (PLM) and can be fine-tuned with only a limited set of dense trajectories. To handle different sampling intervals in sparse trajectories, we first convert each trajectory's sampling interval and movement features into natural language representations, allowing the PLM to recognize its interval. We then introduce a trajectory encoder to unify trajectories of varying intervals into a single interval and capture their spatiotemporal relationships. To obtain road conditions for missing points, we propose an area flow-guided implicit trajectory prompt, which models road conditions by collecting traffic flows in each region. We also introduce a road condition passing mechanism that uses observed points' road conditions to infer those of the missing points. Experiments on two public trajectory datasets with three sampling intervals each demonstrate the effectiveness, scalability, and generalization ability of PLMTrajRec.