mlsanityData to 7 Oct 2026

MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction

arXiv 2111.14973 ↗First posted 29 Nov 2021cs.CV cs.AI cs.LGcited by 246

Not on the map. Why?

Cited by 31 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Predicting the future behavior of road users is one of the most challenging and important problems in autonomous driving. Applying deep learning to this problem requires fusing heterogeneous world state in the form of rich perception signals and map information, and inferring highly multi-modal distributions over possible futures. In this paper, we present MultiPath++, a future prediction model that achieves state-of-the-art performance on popular benchmarks. MultiPath++ improves the MultiPath architecture by revisiting many design choices. The first key design difference is a departure from dense image-based encoding of the input world state in favor of a sparse encoding of heterogeneous scene elements: MultiPath++ consumes compact and efficient polylines to describe road features, and raw agent state information directly (e.g., position, velocity, acceleration). We propose a context-aware fusion of these elements and develop a reusable multi-context gating fusion component. Second, we reconsider the choice of pre-defined, static anchors, and develop a way to learn latent anchor embeddings end-to-end in the model. Lastly, we explore ensembling and output aggregation techniques -- common in other ML domains -- and find effective variants for our probabilistic multimodal output representation. We perform an extensive ablation on these design choices, and show that our proposed model achieves state-of-the-art performance on the Argoverse Motion Forecasting Competition and the Waymo Open Dataset Motion Prediction Challenge.

Key facts. Cited by 246 papers in the map (data through 7 Oct 2026). Builds on 41 of its 51 references that are in the map. First posted on arXiv 29 Nov 2021.

Builds on

41 of 51 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  7. 2017Deep Setscited by 1,394
  8. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  9. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  10. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  11. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  12. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
29 more
  1. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  2. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  3. 2020TNT: Target-driveN Trajectory Predictioncited by 345
  4. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  5. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  6. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  7. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  8. 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
  9. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  10. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  11. 2019Multiple Futures Predictioncited by 178
  12. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  13. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  14. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  15. 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
  16. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  17. 2021TPCN: Temporal Point Cloud Networks for Motion Forecastingcited by 94
  18. 2019Diverse Trajectory Forecasting with Determinantal Point Processescited by 80
  19. 2020What-If Motion Prediction for Autonomous Drivingcited by 80
  20. 2019The Garden of Forking Paths: Towards Multi-Future Trajectory Predictioncited by 72
  21. 2020MANTRA: Memory Augmented Networks for Multiple Trajectory Predictioncited by 69
  22. 2020Diverse and Admissible Trajectory Forecasting through Multimodal Context Understandingcited by 68
  23. 2020Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Datacited by 58
  24. 2021Identifying Driver Interactions via Conditional Behavior Predictioncited by 48
  25. 2020Hierarchical Recurrent Attention Networks for Structured Online Mapscited by 38
  26. 2020PRANK: motion Prediction based on RANKingcited by 22
  27. 2020PLOP: Probabilistic poLynomial Objects trajectory Planning for autonomous drivingcited by 19
  28. 2021DenseTNT: Waymo Open Dataset Motion Prediction Challenge 1st Place Solutioncited by 14
  29. 2019Agent Prioritization for Autonomous Navigationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 37; 2023: 72; 2024: 73; 2025: 37; 2026: 27.

246 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,068
  2. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
  3. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
  4. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  5. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 154
  6. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  7. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  8. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  9. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 109
  10. 2023The Waymo Open Sim Agents Challengecited by 86
  11. 2023Language Conditioned Traffic Generationcited by 86
  12. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 72
38 more
  1. 2022Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Drivingcited by 71
  2. 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
  3. 2022Motion Inspired Unsupervised Perception and Prediction in Autonomous Drivingcited by 59
  4. 2023Trajeglish: Traffic Modeling as Next-Token Predictioncited by 59
  5. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Predictioncited by 56
  6. 2023CAT: Closed-loop Adversarial Training for Safe End-to-End Drivingcited by 52
  7. 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Drivingcited by 52
  8. 2024UniTraj: A Unified Framework for Scalable Vehicle Trajectory Predictioncited by 51
  9. 2023LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraintscited by 50
  10. 2022GANet: Goal Area Network for Motion Forecastingcited by 49
  11. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 49
  12. 2023ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposalscited by 47
  13. 2023Leveraging Future Relationship Reasoning for Vehicle Trajectory Predictioncited by 47
  14. 2024SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rolloutcited by 47
  15. 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusioncited by 46
  16. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 46
  17. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 43
  18. 2023Traj-MAE: Masked Autoencoders for Trajectory Predictioncited by 42
  19. 2022FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphscited by 41
  20. 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
  21. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
  22. 2023ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptationcited by 39
  23. 2024HPNet: Dynamic Trajectory Forecasting with Historical Prediction Attentioncited by 37
  24. 2024SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Predictioncited by 33
  25. 2023MacFormer: Map-Agent Coupled Transformer for Real-time and Robust Trajectory Predictioncited by 32
  26. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 32
  27. 2022SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Drivingcited by 31
  28. 2024WcDT: World-centric Diffusion Transformer for Traffic Scene Generationcited by 31
  29. 2022Behavioral Intention Prediction in Driving Scenes: A Surveycited by 30
  30. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  31. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  32. 2022Safe Real-World Autonomous Driving by Learning to Predict and Plan with a Mixture of Expertscited by 28
  33. 2023Dynamic Scenario Representation Learning for Motion Forecasting with Heterogeneous Graph Convolutional Recurrent Networkscited by 27
  34. 2023SEPT: Towards Efficient Scene Representation Learning for Motion Predictioncited by 27
  35. 2024SIMPL: A Simple and Efficient Multi-agent Motion Prediction Baseline for Autonomous Drivingcited by 27
  36. 2022PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Mapcited by 25
  37. 2022GATraj: A Graph- and Attention-based Multi-Agent Trajectory Prediction Modelcited by 25
  38. 2024CMP: Cooperative Motion Prediction with Multi-Agent Communicationcited by 25

50 most cited of 246 All 246 (JSON)