# Pedestrian Motion Prediction Using Transformer-based Behavior Clustering and Data-Driven Reachability Analysis

arXiv: https://arxiv.org/abs/2408.15250 · HTML: https://mlsanity.com/paper/2408.15250/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 9 Aug 2024.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [Transformer Networks for Trajectory Forecasting](https://mlsanity.com/paper/2003.08111/) (2020)
- [Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction](https://mlsanity.com/paper/2002.08945/) (2020)
- [TITAN: Future Forecast using Action Priors](https://mlsanity.com/paper/2003.13886/) (2020)
- [SIND: A Drone Dataset at Signalized Intersection in China](https://mlsanity.com/paper/2209.02297/) (2022)
- [Pre-training General Trajectory Embeddings with Maximum Multi-view Entropy Coding](https://mlsanity.com/paper/2207.14539/) (2022)
- [VIENA2: A Driving Anticipation Dataset](https://mlsanity.com/paper/1810.09044/) (2018)
- [PSI: A Benchmark for Human Interpretation and Response in Traffic Interactions](https://mlsanity.com/paper/2112.02604/) (2021)
- [Data-Driven Reachability Analysis from Noisy Data](https://mlsanity.com/paper/2105.07229/) (2021)
- [Data-Driven Reachability Analysis with Christoffel Functions](https://mlsanity.com/paper/2104.13902/) (2021)
- [Under the Hood of Transformer Networks for Trajectory Forecasting](https://mlsanity.com/paper/2203.11878/) (2022)
- [Enhancing Data-Driven Reachability Analysis using Temporal Logic Side Information](https://mlsanity.com/paper/2109.07121/) (2021)
- [Data-Driven Reachability Analysis of Pedestrians Using Behavior Modes](https://mlsanity.com/paper/2308.10634/) (2023)

## Built on by (most cited first, 1 of 1)

- [Hazy Pedestrian Trajectory Prediction via Physical Priors and Graph-Mamba](https://mlsanity.com/paper/2509.24020/) (2025)

## Abstract (arXiv)

In this work, we present a transformer-based framework for predicting future pedestrian states based on clustered historical trajectory data. In previous studies, researchers propose enhancing pedestrian trajectory predictions by using manually crafted labels to categorize pedestrian behaviors and intentions. However, these approaches often only capture a limited range of pedestrian behaviors and introduce human bias into the predictions. To alleviate the dependency on manually crafted labels, we utilize a transformer encoder coupled with hierarchical density-based clustering to automatically identify diverse behavior patterns, and use these clusters in data-driven reachability analysis. By using a transformer-based approach, we seek to enhance the representation of pedestrian trajectories and uncover characteristics or features that are subsequently used to group trajectories into different "behavior" clusters. We show that these behavior clusters can be used with data-driven reachability analysis, yielding an end-to-end data-driven approach to predicting the future motion of pedestrians. We train and evaluate our approach on a real pedestrian dataset, showcasing its effectiveness in forecasting pedestrian movements.