# Robustness Benchmark of Road User Trajectory Prediction Models for Automated Driving

arXiv: https://arxiv.org/abs/2304.01895 · HTML: https://mlsanity.com/paper/2304.01895/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 18 references that are in the map.
- First posted on arXiv 4 Apr 2023.

## Builds on (most cited first, 14 of 14)

- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction](https://mlsanity.com/paper/2111.14973/) (2021)
- [Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks](https://mlsanity.com/paper/1907.03395/) (2019)
- [On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles](https://mlsanity.com/paper/2201.05057/) (2022)
- [Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture](https://mlsanity.com/paper/1802.06338/) (2018)
- [AdvDO: Realistic Adversarial Attacks for Trajectory Prediction](https://mlsanity.com/paper/2209.08744/) (2022)
- [Vehicle trajectory prediction works, but not everywhere](https://mlsanity.com/paper/2112.03909/) (2021)
- [Are socially-aware trajectory prediction models really socially-aware?](https://mlsanity.com/paper/2108.10879/) (2021)
- [Pedestrian Trajectory Prediction with Convolutional Neural Networks](https://mlsanity.com/paper/2010.05796/) (2020)
- [MotionCNN: A Strong Baseline for Motion Prediction in Autonomous Driving](https://mlsanity.com/paper/2206.02163/) (2022)
- [Deep Learning for Vision-based Prediction: A Survey](https://mlsanity.com/paper/2007.00095/) (2020)
- [CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationships](https://mlsanity.com/paper/2207.03586/) (2022)
- [MPA: MultiPath++ Based Architecture for Motion Prediction](https://mlsanity.com/paper/2206.10041/) (2022)
- [Scenario-based Evaluation of Prediction Models for Automated Vehicles](https://mlsanity.com/paper/2210.06553/) (2022)

## Built on by (most cited first, 1 of 1)

- [LiMTR: Time Series Motion Prediction for Diverse Road Users through Multimodal Feature Integration](https://mlsanity.com/paper/2410.15819/) (2024)

## Abstract (arXiv)

Accurate and robust trajectory predictions of road users are needed to enable safe automated driving. To do this, machine learning models are often used, which can show erratic behavior when presented with previously unseen inputs. In this work, two environment-aware models (MotionCNN and MultiPath++) and two common baselines (Constant Velocity and an LSTM) are benchmarked for robustness against various perturbations that simulate functional insufficiencies observed during model deployment in a vehicle: unavailability of road information, late detections, and noise. Results show significant performance degradation under the presence of these perturbations, with errors increasing up to +1444.8% in commonly used trajectory prediction evaluation metrics. Training the models with similar perturbations effectively reduces performance degradation, with error increases of up to +87.5%. We argue that despite being an effective mitigation strategy, data augmentation through perturbations during training does not guarantee robustness towards unforeseen perturbations, since identification of all possible on-road complications is unfeasible. Furthermore, degrading the inputs sometimes leads to more accurate predictions, suggesting that the models are unable to learn the true relationships between the different elements in the data.