mlsanityData to 7 Oct 2026

Multiple Futures Prediction

arXiv 1911.00997 ↗First posted 4 Nov 2019cs.LG cs.CV cs.MAcited by 178

Not on the map. Why?

Abstract (arXiv)

Temporal prediction is critical for making intelligent and robust decisions in complex dynamic environments. Motion prediction needs to model the inherently uncertain future which often contains multiple potential outcomes, due to multi-agent interactions and the latent goals of others. Towards these goals, we introduce a probabilistic framework that efficiently learns latent variables to jointly model the multi-step future motions of agents in a scene. Our framework is data-driven and learns semantically meaningful latent variables to represent the multimodal future, without requiring explicit labels. Using a dynamic attention-based state encoder, we learn to encode the past as well as the future interactions among agents, efficiently scaling to any number of agents. Finally, our model can be used for planning via computing a conditional probability density over the trajectories of other agents given a hypothetical rollout of the 'self' agent. We demonstrate our algorithms by predicting vehicle trajectories of both simulated and real data, demonstrating the state-of-the-art results on several vehicle trajectory datasets.

Key facts. Cited by 178 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 51; 2022: 45; 2023: 46; 2024: 21; 2025: 12; 2026: 2.

178 papers.

Sort:

By area:

  1. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  2. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 327
  3. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
  4. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
  5. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
  6. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  7. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  8. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  9. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  10. 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
  11. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  12. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 184
38 more
  1. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  2. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
  3. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 154
  4. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  5. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  6. 2021Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Predictioncited by 131
  7. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  8. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  9. 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 112
  10. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 109
  11. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  12. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  13. 2023The Waymo Open Sim Agents Challengecited by 86
  14. 2022M2I: From Factored Marginal Trajectory Prediction to Interactive Predictioncited by 81
  15. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  16. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  17. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 72
  18. 2021LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agentscited by 64
  19. 2021Learning to Predict Vehicle Trajectories with Model-based Planningcited by 63
  20. 2022ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planningcited by 63
  21. 2021Stepwise Goal-Driven Networks for Trajectory Predictioncited by 62
  22. 2021Trajectory Prediction with Latent Belief Energy-Based Modelcited by 60
  23. 2022MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Predictioncited by 59
  24. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Predictioncited by 56
  25. 2022StretchBEV: Stretching Future Instance Prediction Spatially and Temporallycited by 47
  26. 2023ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposalscited by 47
  27. 2023Leveraging Future Relationship Reasoning for Vehicle Trajectory Predictioncited by 47
  28. 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 43
  29. 2021Human Trajectory Prediction via Counterfactual Analysiscited by 42
  30. 2024Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Followingcited by 41
  31. 2022Social-Implicit: Rethinking Trajectory Prediction Evaluation and The Effectiveness of Implicit Maximum Likelihood Estimationcited by 40
  32. 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
  33. 2023ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptationcited by 39
  34. 2021RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecastingcited by 36
  35. 2022Graph-based Spatial Transformer with Memory Replay for Multi-future Pedestrian Trajectory Predictioncited by 36
  36. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  37. 2021Pylot: A Modular Platform for Exploring Latency-Accuracy Tradeoffs in Autonomous Vehiclescited by 33
  38. 2021Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulationcited by 32

50 most cited of 178 All 178 (JSON)