mlsanityData to 14 Oct 2026

CoverNet: Multimodal Behavior Prediction using Trajectory Sets

arXiv 1911.10298 ↗First posted 23 Nov 2019cs.LG cs.RO stat.MLcited by 259

On the map: World models → World models for autonomous driving

Abstract (arXiv)

We present CoverNet, a new method for multimodal, probabilistic trajectory prediction for urban driving. Previous work has employed a variety of methods, including multimodal regression, occupancy maps, and 1-step stochastic policies. We instead frame the trajectory prediction problem as classification over a diverse set of trajectories. The size of this set remains manageable due to the limited number of distinct actions that can be taken over a reasonable prediction horizon. We structure the trajectory set to a) ensure a desired level of coverage of the state space, and b) eliminate physically impossible trajectories. By dynamically generating trajectory sets based on the agent's current state, we can further improve our method's efficiency. We demonstrate our approach on public, real-world self-driving datasets, and show that it outperforms state-of-the-art methods.

Key facts. Cited by 259 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 1 of its 32 references that are in the map. First posted on arXiv 23 Nov 2019.

Builds on

1 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks18 Sep

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 50; 2022: 55; 2023: 47; 2024: 41; 2025: 36; 2026: 30.

259 papers.

Sort:

By area:

  1. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  2. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  3. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
  4. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 295
  5. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
  6. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 270
  7. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  8. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  9. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  10. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 212
  11. 2021Multimodal Motion Prediction with Stacked Transformerscited by 208
  12. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
38 more
  1. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  2. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 140
  3. 2021Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Predictioncited by 131
  4. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  5. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 117
  6. 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 113
  7. 2021Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversalscited by 113
  8. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  9. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 104
  10. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 103
  11. 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 101
  12. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  13. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  14. 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 96
  15. 2021TPCN: Temporal Point Cloud Networks for Motion Forecastingcited by 94
  16. 2022On Adversarial Robustness of Trajectory Prediction for Autonomous Vehiclescited by 90
  17. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  18. 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Drivingcited by 73
  19. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 73
  20. 2021LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agentscited by 64
  21. 2022ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planningcited by 64
  22. 2021Learning to Predict Vehicle Trajectories with Model-based Planningcited by 63
  23. 2021Stepwise Goal-Driven Networks for Trajectory Predictioncited by 62
  24. 2022Occupancy Flow Fields for Motion Forecasting in Autonomous Drivingcited by 62
  25. 2021Trajectory Prediction with Latent Belief Energy-Based Modelcited by 60
  26. 2025A Survey of World Models for Autonomous Drivingcited by 60
  27. 2022MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Predictioncited by 59
  28. 2022Motion Inspired Unsupervised Perception and Prediction in Autonomous Drivingcited by 59
  29. 2023LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraintscited by 53
  30. 2022GANet: Goal Area Network for Motion Forecastingcited by 49
  31. 2023ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposalscited by 48
  32. 2023Leveraging Future Relationship Reasoning for Vehicle Trajectory Predictioncited by 48
  33. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 47
  34. 2021Three Steps to Multimodal Trajectory Prediction: Modality Clustering, Classification and Synthesiscited by 46
  35. 2021View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrumscited by 45
  36. 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
  37. 2025Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Trainingcited by 39
  38. 2024HPNet: Dynamic Trajectory Forecasting with Historical Prediction Attentioncited by 38

50 most cited of 259 All 259 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion-Based Environment-Aware Trajectory Prediction
  2. 2024Tractable Joint Prediction and Planning over Discrete Behavior Modes for Urban Driving
  3. 2024DriveGPT: Scaling Autoregressive Behavior Models for Driving
  4. 2026Multimodal Action Diffusion for Robust End-to-End Autonomous Driving
  5. 2026Aligning Microscopic Vehicle and Macroscopic Traffic Statistics: Reconstructing Driving Behavior from Partial Data
  6. 2024DeMo: Decoupling Motion Forecasting into Directional Intentions and Dynamic States
  7. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectories
  8. 2023Improving Autonomous Driving Safety with POP: A Framework for Accurate Partially Observed Trajectory Predictions