mlsanityData to 7 Oct 2026

Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks

arXiv 1809.10732 ↗First posted 18 Sep 2018cs.RO cs.CV cs.LGcited by 289

On the map: World models → World models for autonomous driving

Abstract (arXiv)

Autonomous driving presents one of the largest problems that the robotics and artificial intelligence communities are facing at the moment, both in terms of difficulty and potential societal impact. Self-driving vehicles (SDVs) are expected to prevent road accidents and save millions of lives while improving the livelihood and life quality of many more. However, despite large interest and a number of industry players working in the autonomous domain, there still remains more to be done in order to develop a system capable of operating at a level comparable to best human drivers. One reason for this is high uncertainty of traffic behavior and large number of situations that an SDV may encounter on the roads, making it very difficult to create a fully generalizable system. To ensure safe and efficient operations, an autonomous vehicle is required to account for this uncertainty and to anticipate a multitude of possible behaviors of traffic actors in its surrounding. We address this critical problem and present a method to predict multiple possible trajectories of actors while also estimating their probabilities. The method encodes each actor's surrounding context into a raster image, used as input by deep convolutional networks to automatically derive relevant features for the task. Following extensive offline evaluation and comparison to state-of-the-art baselines, the method was successfully tested on SDVs in closed-course tests.

Key facts. Cited by 289 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 0 of its 44 references that are in the map. First posted on arXiv 18 Sep 2018.

Builds on

0 of 44 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 2; 2020: 1; 2021: 63; 2022: 49; 2023: 59; 2024: 61; 2025: 32; 2026: 22.

289 papers.

Sort:

By area:

  1. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  2. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 327
  3. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  4. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
  5. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
  6. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  7. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  8. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
  9. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  10. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  11. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  12. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
38 more
  1. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 154
  2. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  3. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  4. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
  5. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  6. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  7. 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 112
  8. 2021Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversalscited by 112
  9. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 109
  10. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  11. 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 100
  12. 2021Structured Bird's-Eye-View Traffic Scene Understanding from Onboard Imagescited by 98
  13. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  14. 2021TPCN: Temporal Point Cloud Networks for Motion Forecastingcited by 94
  15. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 84
  16. 2022M2I: From Factored Marginal Trajectory Prediction to Interactive Predictioncited by 81
  17. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  18. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  19. 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Drivingcited by 73
  20. 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
  21. 2021LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agentscited by 64
  22. 2021Learning to Predict Vehicle Trajectories with Model-based Planningcited by 63
  23. 2022ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planningcited by 63
  24. 2022Occupancy Flow Fields for Motion Forecasting in Autonomous Drivingcited by 62
  25. 2022Motion Inspired Unsupervised Perception and Prediction in Autonomous Drivingcited by 59
  26. 2019Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safetycited by 48
  27. 2021View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrumscited by 44
  28. 2022Social-Implicit: Rethinking Trajectory Prediction Evaluation and The Effectiveness of Implicit Maximum Likelihood Estimationcited by 40
  29. 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
  30. 2023ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptationcited by 39
  31. 2021Divide-and-Conquer for Lane-Aware Diverse Trajectory Predictioncited by 37
  32. 2024HPNet: Dynamic Trajectory Forecasting with Historical Prediction Attentioncited by 37
  33. 2021Spectral Temporal Graph Neural Network for Trajectory Predictioncited by 36
  34. 2024LC-LLM: Explainable Lane-Change Intention and Trajectory Predictions with Large Language Modelscited by 35
  35. 2021Topology Preserving Local Road Network Estimation from Single Onboard Camera Imagecited by 34
  36. 2023Diverse Human Motion Prediction Guided by Multi-Level Spatial-Temporal Anchorscited by 34
  37. 2024SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Predictioncited by 33
  38. 2021Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulationcited by 32

50 most cited of 289 All 289 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2019Predicting Motion of Vulnerable Road Users using High-Definition Maps and Efficient ConvNets
  2. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey
  3. 2023End-to-end Autonomous Driving using Deep Learning: A Systematic Review
  4. 2026Multi-model approach for autonomous driving: A comprehensive study on traffic sign-, vehicle- and lane detection and behavioral cloning
  5. 2025Large Foundation Models for Trajectory Prediction in Autonomous Driving: A Comprehensive Survey
  6. 2021ObserveNet Control: A Vision-Dynamics Learning Approach to Predictive Control in Autonomous Vehicles
  7. 2023Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models
  8. 2025A Survey of World Models for Autonomous Driving