# Pedestrian Trajectory Prediction via Spatial Interaction Transformer Network

arXiv: https://arxiv.org/abs/2112.06624 · HTML: https://mlsanity.com/paper/2112.06624/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 13 Dec 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2005.08514/) (2020)
- [Transformer Networks for Trajectory Forecasting](https://mlsanity.com/paper/2003.08111/) (2020)
- [TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents](https://mlsanity.com/paper/1811.02146/) (2018)
- [GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/1907.07792/) (2019)
- [Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty](https://mlsanity.com/paper/1711.09026/) (2017)
- [Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture](https://mlsanity.com/paper/1802.06338/) (2018)
- [Top-view Trajectories: A Pedestrian Dataset of Vehicle-Crowd Interaction from Controlled Experiments and Crowded Campus](https://mlsanity.com/paper/1902.00487/) (2019)
- [Deep Learning for Vision-based Prediction: A Survey](https://mlsanity.com/paper/2007.00095/) (2020)
- [Multimodal Future Localization and Emergence Prediction for Objects in Egocentric View with a Reachability Prior](https://mlsanity.com/paper/2006.04700/) (2020)
- [Forecasting Pedestrian Trajectory with Machine-Annotated Training Data](https://mlsanity.com/paper/1905.03681/) (2019)

## Built on by (most cited first, 4 of 4)

- [Human Trajectory Prediction via Neural Social Physics](https://mlsanity.com/paper/2207.10435/) (2022)
- [Under the Hood of Transformer Networks for Trajectory Forecasting](https://mlsanity.com/paper/2203.11878/) (2022)
- [Vulnerable Road User Detection and Safety Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2405.19202/) (2024)
- [Predicting Road Crossing Behaviour using Pose Detection and Sequence Modelling](https://mlsanity.com/paper/2508.15336/) (2025)

## Abstract (arXiv)

As a core technology of the autonomous driving system, pedestrian trajectory prediction can significantly enhance the function of active vehicle safety and reduce road traffic injuries. In traffic scenes, when encountering with oncoming people, pedestrians may make sudden turns or stop immediately, which often leads to complicated trajectories. To predict such unpredictable trajectories, we can gain insights into the interaction between pedestrians. In this paper, we present a novel generative method named Spatial Interaction Transformer (SIT), which learns the spatio-temporal correlation of pedestrian trajectories through attention mechanisms. Furthermore, we introduce the conditional variational autoencoder (CVAE) framework to model the future latent motion states of pedestrians. In particular, the experiments based on large-scale trafc dataset nuScenes [2] show that SIT has an outstanding performance than state-of-the-art (SOTA) methods. Experimental evaluation on the challenging ETH and UCY datasets conrms the robustness of our proposed model