# A Critical Perceptual Pre-trained Model for Complex Trajectory Recovery

arXiv: https://arxiv.org/abs/2311.02631 · HTML: https://mlsanity.com/paper/2311.02631/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 5 Nov 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- [Do Transformers Really Perform Bad for Graph Representation?](https://mlsanity.com/paper/2106.05234/) (2021)
- [Unsupervised Path Representation Learning with Curriculum Negative Sampling](https://mlsanity.com/paper/2106.09373/) (2021)
- [Jointly Contrastive Representation Learning on Road Network and Trajectory](https://mlsanity.com/paper/2209.06389/) (2022)
- [Tensor Completion for Weakly-dependent Data on Graph for Metro Passenger Flow Prediction](https://mlsanity.com/paper/1912.05693/) (2019)
- [Trajectory Recovery From Ash: User Privacy Is NOT Preserved in Aggregated Mobility Data](https://mlsanity.com/paper/1702.06270/) (2017)
- [Long-Short Term Spatiotemporal Tensor Prediction for Passenger Flow Profile](https://mlsanity.com/paper/2004.11022/) (2020)
- [DeepMove: Learning Place Representations through Large Scale Movement Data](https://mlsanity.com/paper/1807.04241/) (2018)
- [Trajectory Forecasting from Detection with Uncertainty-Aware Motion Encoding](https://mlsanity.com/paper/2202.01478/) (2022)

## Built on by (most cited first, 4 of 4)

- [VisionTraj: A Noise-Robust Trajectory Recovery Framework based on Large-scale Camera Network](https://mlsanity.com/paper/2312.06428/) (2023)
- [Skin Cancer Detection utilizing Deep Learning: Classification of Skin Lesion Images using a Vision Transformer](https://mlsanity.com/paper/2407.18554/) (2024)
- [Transformer-based Drum-level Prediction in a Boiler Plant with Delayed Relations among Multivariates](https://mlsanity.com/paper/2407.11180/) (2024)
- [Exploring Genre and Success Classification through Song Lyrics using DistilBERT: A Fun NLP Venture](https://mlsanity.com/paper/2407.21068/) (2024)

## Abstract (arXiv)

The trajectory on the road traffic is commonly collected at a low sampling rate, and trajectory recovery aims to recover a complete and continuous trajectory from the sparse and discrete inputs. Recently, sequential language models have been innovatively adopted for trajectory recovery in a pre-trained manner: it learns road segment representation vectors, which will be used in the downstream tasks. However, existing methods are incapable of handling complex trajectories: when the trajectory crosses remote road segments or makes several turns, which we call critical nodes, the quality of learned representations deteriorates, and the recovered trajectories skip the critical nodes. This work is dedicated to offering a more robust trajectory recovery for complex trajectories. Firstly, we define the trajectory complexity based on the detour score and entropy score and construct the complexity-aware semantic graphs correspondingly. Then, we propose a Multi-view Graph and Complexity Aware Transformer (MGCAT) model to encode these semantics in trajectory pre-training from two aspects: 1) adaptively aggregate the multi-view graph features considering trajectory pattern, and 2) higher attention to critical nodes in a complex trajectory. Such that, our MGCAT is perceptual when handling the critical scenario of complex trajectories. Extensive experiments are conducted on large-scale datasets. The results prove that our method learns better representations for trajectory recovery, with 5.22% higher F1-score overall and 8.16% higher F1-score for complex trajectories particularly. The code is available at https://github.com/bonaldli/ComplexTraj.