# Comparison of Pedestrian Prediction Models from Trajectory and Appearance Data for Autonomous Driving

arXiv: https://arxiv.org/abs/2305.15942 · HTML: https://mlsanity.com/paper/2305.15942/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 25 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 16 of 16)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020)
- [It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction](https://mlsanity.com/paper/2004.02025/) (2020)
- [What the Constant Velocity Model Can Teach Us About Pedestrian Motion Prediction](https://mlsanity.com/paper/1903.07933/) (2019)
- [TITAN: Future Forecast using Action Priors](https://mlsanity.com/paper/2003.13886/) (2020)
- [Pedestrian Action Anticipation using Contextual Feature Fusion in Stacked RNNs](https://mlsanity.com/paper/2005.06582/) (2020)
- [Bifold and Semantic Reasoning for Pedestrian Behavior Prediction](https://mlsanity.com/paper/2012.03298/) (2020)
- [Agreeing to Cross: How Drivers and Pedestrians Communicate](https://mlsanity.com/paper/1702.03555/) (2017)
- [Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting](https://mlsanity.com/paper/2101.02385/) (2021)
- [Flash: Fast and Light Motion Prediction for Autonomous Driving with Bayesian Inverse Planning and Learned Motion Profiles](https://mlsanity.com/paper/2203.08251/) (2022)
- [DiPA: Probabilistic Multi-Modal Interactive Prediction for Autonomous Driving](https://mlsanity.com/paper/2210.06106/) (2022)
- [CovarianceNet: Conditional Generative Model for Correct Covariance Prediction in Human Motion Prediction](https://mlsanity.com/paper/2109.02965/) (2021)
- [Early Start Intention Detection of Cyclists Using Motion History Images and a Deep Residual Network](https://mlsanity.com/paper/1803.02242/) (2018)

## Abstract (arXiv)

The ability to anticipate pedestrian motion changes is a critical capability for autonomous vehicles. In urban environments, pedestrians may enter the road area and create a high risk for driving, and it is important to identify these cases. Typical predictors use the trajectory history to predict future motion, however in cases of motion initiation, motion in the trajectory may only be clearly visible after a delay, which can result in the pedestrian has entered the road area before an accurate prediction can be made. Appearance data includes useful information such as changes of gait, which are early indicators of motion changes, and can inform trajectory prediction. This work presents a comparative evaluation of trajectory-only and appearance-based methods for pedestrian prediction, and introduces a new dataset experiment for prediction using appearance. We create two trajectory and image datasets based on the combination of image and trajectory sequences from the popular NuScenes dataset, and examine prediction of trajectories using observed appearance to influence futures. This shows some advantages over trajectory prediction alone, although problems with the dataset prevent advantages of appearance-based models from being shown. We describe methods for improving the dataset and experiment to allow benefits of appearance-based models to be captured.