# Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding

arXiv: https://arxiv.org/abs/2003.03212 · HTML: https://mlsanity.com/paper/2003.03212/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2020.

## Built on by (most cited first, 30 of 68)

- [Motion Transformer with Global Intention Localization and Local Movement Refinement](https://mlsanity.com/paper/2209.13508/) (2022)
- [MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction](https://mlsanity.com/paper/2111.14973/) (2021)
- [Scene Transformer: A unified architecture for predicting multiple agent trajectories](https://mlsanity.com/paper/2106.08417/) (2021)
- [MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Querying](https://mlsanity.com/paper/2306.17770/) (2023)
- [LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving](https://mlsanity.com/paper/2101.06547/) (2021)
- [LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agents](https://mlsanity.com/paper/2104.00249/) (2021)
- [Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection](https://mlsanity.com/paper/2307.07205/) (2023)
- [FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphs](https://mlsanity.com/paper/2211.16197/) (2022)
- [ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptation](https://mlsanity.com/paper/2307.14187/) (2023)
- [Vehicle trajectory prediction works, but not everywhere](https://mlsanity.com/paper/2112.03909/) (2021)
- [On Transforming Reinforcement Learning by Transformer: The Development Trajectory](https://mlsanity.com/paper/2212.14164/) (2022)
- [Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking](https://mlsanity.com/paper/2102.09117/) (2021)
- [Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding](https://mlsanity.com/paper/2310.12970/) (2023)
- [MacFormer: Map-Agent Coupled Transformer for Real-time and Robust Trajectory Prediction](https://mlsanity.com/paper/2308.10280/) (2023)
- [Core Challenges in Embodied Vision-Language Planning](https://mlsanity.com/paper/2106.13948/) (2021)
- [Behavioral Intention Prediction in Driving Scenes: A Survey](https://mlsanity.com/paper/2211.00385/) (2022)
- [The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review](https://mlsanity.com/paper/2308.05731/) (2023)
- [NRTSI: Non-Recurrent Time Series Imputation](https://mlsanity.com/paper/2102.03340/) (2021)
- [R-Pred: Two-Stage Motion Prediction Via Tube-Query Attention-Based Trajectory Refinement](https://mlsanity.com/paper/2211.08609/) (2022)
- [Learn-to-Race: A Multimodal Control Environment for Autonomous Racing](https://mlsanity.com/paper/2103.11575/) (2021)
- [Causal Navigation by Continuous-time Neural Networks](https://mlsanity.com/paper/2106.08314/) (2021)
- [CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning](https://mlsanity.com/paper/2106.13364/) (2021)
- [Path-Aware Graph Attention for HD Maps in Motion Prediction](https://mlsanity.com/paper/2202.13772/) (2022)
- [TrajGen: Generating Realistic and Diverse Trajectories with Reactive and Feasible Agent Behaviors for Autonomous Driving](https://mlsanity.com/paper/2203.16792/) (2022)
- [PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learning](https://mlsanity.com/paper/2210.07886/) (2022)
- [End-to-End Trajectory Distribution Prediction Based on Occupancy Grid Maps](https://mlsanity.com/paper/2203.16910/) (2022)
- [LatentFormer: Multi-Agent Transformer-Based Interaction Modeling and Trajectory Prediction](https://mlsanity.com/paper/2203.01880/) (2022)
- [DICE: Diverse Diffusion Model with Scoring for Trajectory Prediction](https://mlsanity.com/paper/2310.14570/) (2023)
- [Unlimited Neighborhood Interaction for Heterogeneous Trajectory Prediction](https://mlsanity.com/paper/2108.00238/) (2021)
- [Euro-PVI: Pedestrian Vehicle Interactions in Dense Urban Centers](https://mlsanity.com/paper/2106.12442/) (2021)

## Abstract (arXiv)

Multi-agent trajectory forecasting in autonomous driving requires an agent to accurately anticipate the behaviors of the surrounding vehicles and pedestrians, for safe and reliable decision-making. Due to partial observability in these dynamical scenes, directly obtaining the posterior distribution over future agent trajectories remains a challenging problem. In realistic embodied environments, each agent's future trajectories should be both diverse since multiple plausible sequences of actions can be used to reach its intended goals, and admissible since they must obey physical constraints and stay in drivable areas. In this paper, we propose a model that synthesizes multiple input signals from the multimodal world|the environment's scene context and interactions between multiple surrounding agents|to best model all diverse and admissible trajectories. We compare our model with strong baselines and ablations across two public datasets and show a significant performance improvement over previous state-of-the-art methods. Lastly, we offer new metrics incorporating admissibility criteria to further study and evaluate the diversity of predictions. Codes are at: https://github.com/kami93/CMU-DATF.