mlsanityData to 7 Oct 2026

Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals

arXiv 2106.15004 ↗First posted 28 Jun 2021cs.CV cs.ROcited by 112

Not on the map. Why?

Abstract (arXiv)

Accurately predicting the future motion of surrounding vehicles requires reasoning about the inherent uncertainty in driving behavior. This uncertainty can be loosely decoupled into lateral (e.g., keeping lane, turning) and longitudinal (e.g., accelerating, braking). We present a novel method that combines learned discrete policy rollouts with a focused decoder on subsets of the lane graph. The policy rollouts explore different goals given current observations, ensuring that the model captures lateral variability. Longitudinal variability is captured by our latent variable model decoder that is conditioned on various subsets of the lane graph. Our model achieves state-of-the-art performance on the nuScenes motion prediction dataset, and qualitatively demonstrates excellent scene compliance. Detailed ablations highlight the importance of the policy rollouts and the decoder architecture.

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). Builds on 27 of its 36 references that are in the map. First posted on arXiv 28 Jun 2021.

Builds on

27 of 36 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  6. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  7. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  8. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  9. 2020TNT: Target-driveN Trajectory Predictioncited by 345
  10. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  11. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  12. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
15 more
  1. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
  2. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  3. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 235
  4. 2020Long-term Human Motion Prediction with Scene Contextcited by 191
  5. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  6. 2019Multi-Agent Tensor Fusion for Contextual Trajectory Predictioncited by 131
  7. 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 112
  8. 2019Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Predictioncited by 96
  9. 2020What-If Motion Prediction for Autonomous Drivingcited by 80
  10. 2020Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Planscited by 72
  11. 2021Stepwise Goal-Driven Networks for Trajectory Predictioncited by 62
  12. 2020Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Datacited by 58
  13. 2020Map-Adaptive Goal-Based Trajectory Predictioncited by 29
  14. 2020Probabilistic Multi-modal Trajectory Prediction with Lane Attention for Autonomous Vehiclescited by 28
  15. 2020Trajectory Prediction for Autonomous Driving based on Multi-Head Attention with Joint Agent-Map Representationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 17; 2023: 38; 2024: 26; 2025: 20; 2026: 9.

112 papers.

Sort:

By area:

  1. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  2. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  3. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 109
  4. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  5. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 72
  6. 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
  7. 2024NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Drivingcited by 61
  8. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Predictioncited by 56
  9. 2023LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraintscited by 50
  10. 2022AdvDO: Realistic Adversarial Attacks for Trajectory Predictioncited by 48
  11. 2023Leveraging Future Relationship Reasoning for Vehicle Trajectory Predictioncited by 47
  12. 2023Online Vectorized HD Map Construction using Geometrycited by 42
38 more
  1. 2022FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphscited by 41
  2. 2024HPNet: Dynamic Trajectory Forecasting with Historical Prediction Attentioncited by 37
  3. 2023MacFormer: Map-Agent Coupled Transformer for Real-time and Robust Trajectory Predictioncited by 32
  4. 2024Producing and Leveraging Online Map Uncertainty in Trajectory Predictioncited by 32
  5. 2023Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous drivingcited by 30
  6. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  7. 2022PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Mapcited by 25
  8. 2022GATraj: A Graph- and Attention-based Multi-Agent Trajectory Prediction Modelcited by 25
  9. 2022R-Pred: Two-Stage Motion Prediction Via Tube-Query Attention-Based Trajectory Refinementcited by 23
  10. 2022End-to-End Trajectory Distribution Prediction Based on Occupancy Grid Mapscited by 22
  11. 2023RaTrack: Moving Object Detection and Tracking with 4D Radar Point Cloudcited by 21
  12. 2023DICE: Diverse Diffusion Model with Scoring for Trajectory Predictioncited by 18
  13. 2024VisionTrap: Vision-Augmented Trajectory Prediction Guided by Textual Descriptionscited by 18
  14. 2023RS2G: Data-Driven Scene-Graph Extraction and Embedding for Robust Autonomous Perception and Scenario Understandingcited by 17
  15. 2023Human-centric Scene Understanding for 3D Large-scale Scenarioscited by 17
  16. 2024DeTra: A Unified Model for Object Detection and Trajectory Forecastingcited by 15
  17. 2024Diffusion-Based Environment-Aware Trajectory Predictioncited by 14
  18. 2024doScenes: An Autonomous Driving Dataset with Natural Language Instruction for Human Interaction and Vision-Language Navigationcited by 14
  19. 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directionscited by 14
  20. 2022Uncertainty estimation for Cross-dataset performance in Trajectory predictioncited by 13
  21. 2022Collaborative Uncertainty Benefits Multi-Agent Multi-Modal Trajectory Forecastingcited by 13
  22. 2023Context-Aware Timewise VAEs for Real-Time Vehicle Trajectory Predictioncited by 12
  23. 2024NEST: A Neuromodulated Small-world Hypergraph Trajectory Prediction Model for Autonomous Drivingcited by 12
  24. 2022HOPE: Hierarchical Spatial-temporal Network for Occupancy Flow Predictioncited by 11
  25. 2024Intention-aware Denoising Diffusion Model for Trajectory Predictioncited by 11
  26. 2024Motion Forecasting in Continuous Drivingcited by 11
  27. 2022Towards Explainable Motion Prediction using Heterogeneous Graph Representationscited by 10
  28. 2023VT-Former: An Exploratory Study on Vehicle Trajectory Prediction for Highway Surveillance through Graph Isomorphism and Transformercited by 10
  29. 2024SemanticFormer: Holistic and Semantic Traffic Scene Representation for Trajectory Prediction using Knowledge Graphscited by 10
  30. 2022Aware of the History: Trajectory Forecasting with the Local Behavior Datacited by 8
  31. 2022Exploiting map information for self-supervised learning in motion forecastingcited by 8
  32. 2024Accelerating Online Mapping and Behavior Prediction via Direct BEV Feature Attentioncited by 8
  33. 2025Motion Forecasting for Autonomous Vehicles: A Surveycited by 8
  34. 2023Offline Tracking with Object Permanencecited by 7
  35. 2023Conditional Unscented Autoencoders for Trajectory Predictioncited by 7
  36. 2025GoIRL: Graph-Oriented Inverse Reinforcement Learning for Multimodal Trajectory Predictioncited by 7
  37. 2023Pix2Map: Cross-modal Retrieval for Inferring Street Maps from Imagescited by 6
  38. 2023PBP: Path-based Trajectory Prediction for Autonomous Drivingcited by 6

50 most cited of 112 All 112 (JSON)