# Benchmark for Models Predicting Human Behavior in Gap Acceptance Scenarios

arXiv: https://arxiv.org/abs/2211.05455 · HTML: https://mlsanity.com/paper/2211.05455/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 65 references that are in the map.
- First posted on arXiv 10 Nov 2022.

## Builds on (most cited first, 28 of 28)

- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints](https://mlsanity.com/paper/1806.01482/) (2018)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence](https://mlsanity.com/paper/2002.06177/) (2020)
- [Transformer Networks for Trajectory Forecasting](https://mlsanity.com/paper/2003.08111/) (2020)
- [VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change](https://mlsanity.com/paper/2005.08135/) (2020)
- [LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving](https://mlsanity.com/paper/2101.06547/) (2021)
- [On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles](https://mlsanity.com/paper/2201.05057/) (2022)
- [Human Trajectory Prediction via Neural Social Physics](https://mlsanity.com/paper/2207.10435/) (2022)
- [AdvDO: Realistic Adversarial Attacks for Trajectory Prediction](https://mlsanity.com/paper/2209.08744/) (2022)
- [Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention](https://mlsanity.com/paper/2104.05485/) (2021)
- [Interpretable Social Anchors for Human Trajectory Forecasting in Crowds](https://mlsanity.com/paper/2105.03136/) (2021)
- [Graph Neural Networks for Modelling Traffic Participant Interaction](https://mlsanity.com/paper/1903.01254/) (2019)
- [Modular Design Patterns for Hybrid Learning and Reasoning Systems: a taxonomy, patterns and use cases](https://mlsanity.com/paper/2102.11965/) (2021)
- [Deep Learning-based Vehicle Behaviour Prediction For Autonomous Driving Applications: A Review](https://mlsanity.com/paper/1912.11676/) (2019)
- [Pedestrian Models for Autonomous Driving Part II: High-Level Models of Human Behavior](https://mlsanity.com/paper/2003.11959/) (2020)
- [Injecting Planning-Awareness into Prediction and Detection Evaluation](https://mlsanity.com/paper/2110.03270/) (2021)
- [Causal-based Time Series Domain Generalization for Vehicle Intention Prediction](https://mlsanity.com/paper/2112.02093/) (2021)
- [HYPER: Learned Hybrid Trajectory Prediction via Factored Inference and Adaptive Sampling](https://mlsanity.com/paper/2110.02344/) (2021)
- [Using Graph-Theoretic Machine Learning to Predict Human Driver Behavior](https://mlsanity.com/paper/2111.02964/) (2021)
- [Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles](https://mlsanity.com/paper/2207.12380/) (2022)
- [On complementing end-to-end human behavior predictors with planning](https://mlsanity.com/paper/2103.05661/) (2021)
- [Early Lane Change Prediction for Automated Driving Systems Using Multi-Task Attention-based Convolutional Neural Networks](https://mlsanity.com/paper/2109.10742/) (2021)
- [Leveraging Smooth Attention Prior for Multi-Agent Trajectory Prediction](https://mlsanity.com/paper/2203.04421/) (2022)
- [Comparing merging behaviors observed in naturalistic data with behaviors generated by a machine learned model](https://mlsanity.com/paper/2104.10496/) (2021)
- [Predicting highway lane-changing maneuvers: A benchmark analysis of machine and ensemble learning algorithms](https://mlsanity.com/paper/2204.10807/) (2022)
- [Beyond RMSE: Do machine-learned models of road user interaction produce human-like behavior?](https://mlsanity.com/paper/2206.11110/) (2022)

## Built on by (most cited first, 5 of 5)

- [Using Models Based on Cognitive Theory to Predict Human Behavior in Traffic: A Case Study](https://mlsanity.com/paper/2305.15187/) (2023)
- [Realistic Adversarial Attacks for Robustness Evaluation of Trajectory Prediction Models via Future State Perturbation](https://mlsanity.com/paper/2505.06134/) (2025)
- [STEP: Structured Training and Evaluation Platform for benchmarking trajectory prediction models](https://mlsanity.com/paper/2509.14801/) (2025)
- [Smooth-Trajectron++: Augmenting the Trajectron++ behaviour prediction model with smooth attention](https://mlsanity.com/paper/2305.19678/) (2023)
- [Evaluating randomized smoothing as a defense against adversarial attacks in trajectory prediction](https://mlsanity.com/paper/2603.10821/) (2026)

## Abstract (arXiv)

Autonomous vehicles currently suffer from a time-inefficient driving style caused by uncertainty about human behavior in traffic interactions. Accurate and reliable prediction models enabling more efficient trajectory planning could make autonomous vehicles more assertive in such interactions. However, the evaluation of such models is commonly oversimplistic, ignoring the asymmetric importance of prediction errors and the heterogeneity of the datasets used for testing. We examine the potential of recasting interactions between vehicles as gap acceptance scenarios and evaluating models in this structured environment. To that end, we develop a framework aiming to facilitate the evaluation of any model, by any metric, and in any scenario. We then apply this framework to state-of-the-art prediction models, which all show themselves to be unreliable in the most safety-critical situations.