# What-If Motion Prediction for Autonomous Driving

arXiv: https://arxiv.org/abs/2008.10587 · HTML: https://mlsanity.com/paper/2008.10587/

## Key facts

- Cited by 80 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2020.

## Built on by (most cited first, 30 of 80)

- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021)
- [Wayformer: Motion Forecasting via Simple & Efficient Attention Networks](https://mlsanity.com/paper/2207.05844/) (2022)
- [MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction](https://mlsanity.com/paper/2111.14973/) (2021)
- [MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion](https://mlsanity.com/paper/2306.03083/) (2023)
- [Multimodal Motion Prediction with Stacked Transformers](https://mlsanity.com/paper/2103.11624/) (2021)
- [Scene Transformer: A unified architecture for predicting multiple agent trajectories](https://mlsanity.com/paper/2106.08417/) (2021)
- [MotionLM: Multi-Agent Motion Forecasting as Language Modeling](https://mlsanity.com/paper/2309.16534/) (2023)
- [GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation](https://mlsanity.com/paper/2109.01827/) (2021)
- [Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction](https://mlsanity.com/paper/2104.00563/) (2021)
- [HOME: Heatmap Output for future Motion Estimation](https://mlsanity.com/paper/2105.10968/) (2021)
- [LaneRCNN: Distributed Representations for Graph-Centric Motion Forecasting](https://mlsanity.com/paper/2101.06653/) (2021)
- [Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals](https://mlsanity.com/paper/2106.15004/) (2021)
- [THOMAS: Trajectory Heatmap Output with learned Multi-Agent Sampling](https://mlsanity.com/paper/2110.06607/) (2021)
- [M2I: From Factored Marginal Trajectory Prediction to Interactive Prediction](https://mlsanity.com/paper/2202.11884/) (2022)
- [Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoders](https://mlsanity.com/paper/2308.09882/) (2023)
- [LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agents](https://mlsanity.com/paper/2104.00249/) (2021)
- [LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraints](https://mlsanity.com/paper/2302.13933/) (2023)
- [ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptation](https://mlsanity.com/paper/2307.14187/) (2023)
- [JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving](https://mlsanity.com/paper/2212.08710/) (2022)
- [Vehicle trajectory prediction works, but not everywhere](https://mlsanity.com/paper/2112.03909/) (2021)
- [Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction](https://mlsanity.com/paper/2104.08277/) (2021)
- [Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future](https://mlsanity.com/paper/2512.16760/) (2025)
- [SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Driving](https://mlsanity.com/paper/2206.14116/) (2022)
- [The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review](https://mlsanity.com/paper/2308.05731/) (2023)
- [Dynamic Scenario Representation Learning for Motion Forecasting with Heterogeneous Graph Convolutional Recurrent Networks](https://mlsanity.com/paper/2303.04364/) (2023)
- [WOMD-LiDAR: Raw Sensor Dataset Benchmark for Motion Forecasting](https://mlsanity.com/paper/2304.03834/) (2023)
- [CRAT-Pred: Vehicle Trajectory Prediction with Crystal Graph Convolutional Neural Networks and Multi-Head Self-Attention](https://mlsanity.com/paper/2202.04488/) (2022)
- [CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationships](https://mlsanity.com/paper/2207.03586/) (2022)
- [SCENE: Reasoning about Traffic Scenes using Heterogeneous Graph Neural Networks](https://mlsanity.com/paper/2301.03512/) (2023)

## Abstract (arXiv)

Forecasting the long-term future motion of road actors is a core challenge to the deployment of safe autonomous vehicles (AVs). Viable solutions must account for both the static geometric context, such as road lanes, and dynamic social interactions arising from multiple actors. While recent deep architectures have achieved state-of-the-art performance on distance-based forecasting metrics, these approaches produce forecasts that are predicted without regard to the AV's intended motion plan. In contrast, we propose a recurrent graph-based attentional approach with interpretable geometric (actor-lane) and social (actor-actor) relationships that supports the injection of counterfactual geometric goals and social contexts. Our model can produce diverse predictions conditioned on hypothetical or "what-if" road lanes and multi-actor interactions. We show that such an approach could be used in the planning loop to reason about unobserved causes or unlikely futures that are directly relevant to the AV's intended route.