# Mode Collapse Happens: Evaluating Critical Interactions in Joint Trajectory Prediction Models

arXiv: https://arxiv.org/abs/2506.23164 · HTML: https://mlsanity.com/paper/2506.23164/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 29 Jun 2025.

## Builds on (most cited first, 18 of 18)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021)
- [DLow: Diversifying Latent Flows for Diverse Human Motion Prediction](https://mlsanity.com/paper/2003.08386/) (2020)
- [The inD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersections](https://mlsanity.com/paper/1911.07602/) (2019)
- [Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories with GANs](https://mlsanity.com/paper/1904.09507/) (2019)
- [Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Plans](https://mlsanity.com/paper/2001.00735/) (2020)
- [ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planning](https://mlsanity.com/paper/2206.13387/) (2022)
- [JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving](https://mlsanity.com/paper/2212.08710/) (2022)
- [The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review](https://mlsanity.com/paper/2308.05731/) (2023)
- [Search-based optimal motion planning for automated driving](https://mlsanity.com/paper/1803.04868/) (2018)
- [Tree-structured Policy Planning with Learned Behavior Models](https://mlsanity.com/paper/2301.11902/) (2023)
- [Interaction-Based Trajectory Prediction Over a Hybrid Traffic Graph](https://mlsanity.com/paper/2009.12916/) (2020)
- [Interactive Joint Planning for Autonomous Vehicles](https://mlsanity.com/paper/2310.18301/) (2023)
- [Categorical Traffic Transformer: Interpretable and Diverse Behavior Prediction with Tokenized Latent](https://mlsanity.com/paper/2311.18307/) (2023)

## Abstract (arXiv)

Autonomous Vehicle decisions rely on multimodal prediction models that account for multiple route options and the inherent uncertainty in human behavior. However, models can suffer from mode collapse, where only the most likely mode is predicted, posing significant safety risks. While existing methods employ various strategies to generate diverse predictions, they often overlook the diversity in interaction modes among agents. Additionally, traditional metrics for evaluating prediction models are dataset-dependent and do not evaluate inter-agent interactions quantitatively. To our knowledge, none of the existing metrics explicitly evaluates mode collapse. In this paper, we propose a novel evaluation framework that assesses mode collapse in joint trajectory predictions, focusing on safety-critical interactions. We introduce metrics for mode collapse, mode correctness, and coverage, emphasizing the sequential dimension of predictions. By testing four multi-agent trajectory prediction models, we demonstrate that mode collapse indeed happens. When looking at the sequential dimension, although prediction accuracy improves closer to interaction events, there are still cases where the models are unable to predict the correct interaction mode, even just before the interaction mode becomes inevitable. We hope that our framework can help researchers gain new insights and advance the development of more consistent and accurate prediction models, thus enhancing the safety of autonomous driving systems.