# AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points

arXiv: https://arxiv.org/abs/2007.05719 · HTML: https://mlsanity.com/paper/2007.05719/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2020.

## Built on by (most cited first, 6 of 6)

- [EigenTrajectory: Low-Rank Descriptors for Multi-Modal Trajectory Forecasting](https://mlsanity.com/paper/2307.09306/) (2023)
- [SingularTrajectory: Universal Trajectory Predictor Using Diffusion Model](https://mlsanity.com/paper/2403.18452/) (2024)
- [Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction](https://mlsanity.com/paper/2403.18447/) (2024)
- [Towards Robust Human Trajectory Prediction in Raw Videos](https://mlsanity.com/paper/2108.08259/) (2021)
- [Continuous Locomotive Crowd Behavior Generation](https://mlsanity.com/paper/2504.04756/) (2025)
- [TrajPRed: Trajectory Prediction with Region-based Relation Learning](https://mlsanity.com/paper/2404.06971/) (2024)

## Abstract (arXiv)

Current methods for trajectory prediction operate in supervised manners, and therefore require vast quantities of corresponding ground truth data for training. In this paper, we present a novel, label-free algorithm, AutoTrajectory, for trajectory extraction and prediction to use raw videos directly. To better capture the moving objects in videos, we introduce dynamic points. We use them to model dynamic motions by using a forward-backward extractor to keep temporal consistency and using image reconstruction to keep spatial consistency in an unsupervised manner. Then we aggregate dynamic points to instance points, which stand for moving objects such as pedestrians in videos. Finally, we extract trajectories by matching instance points for prediction training. To the best of our knowledge, our method is the first to achieve unsupervised learning of trajectory extraction and prediction. We evaluate the performance on well-known trajectory datasets and show that our method is effective for real-world videos and can use raw videos to further improve the performance of existing models.