mlsanityData to 7 Oct 2026

Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net

arXiv 2012.12395 ↗First posted 22 Dec 2020cs.CVcited by 228

Not on the map. Why?

Cited by 32 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper we propose a novel deep neural network that is able to jointly reason about 3D detection, tracking and motion forecasting given data captured by a 3D sensor. By jointly reasoning about these tasks, our holistic approach is more robust to occlusion as well as sparse data at range. Our approach performs 3D convolutions across space and time over a bird's eye view representation of the 3D world, which is very efficient in terms of both memory and computation. Our experiments on a new very large scale dataset captured in several north american cities, show that we can outperform the state-of-the-art by a large margin. Importantly, by sharing computation we can perform all tasks in as little as 30 ms.

Key facts. Cited by 228 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 60; 2022: 51; 2023: 52; 2024: 34; 2025: 17; 2026: 14.

228 papers.

Sort:

By area:

  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  2. 2022Planning-oriented Autonomous Drivingcited by 1,068
  3. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  4. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  5. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  6. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  7. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  8. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 239
  9. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  10. 2021Learning Distilled Collaboration Graph for Multi-Agent Perceptioncited by 194
  11. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  12. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
38 more
  1. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  2. 2022V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Drivingcited by 157
  3. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  4. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  5. 2022Learning from All Vehiclescited by 135
  6. 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 132
  7. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  8. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  9. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusioncited by 105
  10. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  11. 2021TPCN: Temporal Point Cloud Networks for Motion Forecastingcited by 94
  12. 2021FusionPainting: Multimodal Fusion with Adaptive Attention for 3D Object Detectioncited by 92
  13. 2023Collaborative Perception in Autonomous Driving: Methods, Datasets and Challengescited by 87
  14. 2021Space-Time-Separable Graph Convolutional Network for Pose Forecastingcited by 84
  15. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  16. 20213D-MAN: 3D Multi-frame Attention Network for Object Detectioncited by 73
  17. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  18. 2022PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequencescited by 68
  19. 2021Box-Aware Feature Enhancement for Single Object Tracking on Point Cloudscited by 66
  20. 20223D Object Detection from Images for Autonomous Driving: A Surveycited by 66
  21. 2022MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detectioncited by 65
  22. 2023Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perceptioncited by 65
  23. 2022Occupancy Flow Fields for Motion Forecasting in Autonomous Drivingcited by 62
  24. 2021PTT: Point-Track-Transformer Module for 3D Single Object Tracking in Point Cloudscited by 59
  25. 2022Motion Inspired Unsupervised Perception and Prediction in Autonomous Drivingcited by 59
  26. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  27. 2021Exploring Simple 3D Multi-Object Tracking for Autonomous Drivingcited by 56
  28. 20213D Siamese Voxel-to-BEV Tracker for Sparse Point Cloudscited by 54
  29. 2023Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Trackingcited by 54
  30. 2021RelationTrack: Relation-aware Multiple Object Tracking with Decoupled Representationcited by 52
  31. 2021Scalable Scene Flow from Point Clouds in the Real Worldcited by 49
  32. 2023HiLM-D: Enhancing MLLMs with Multi-Scale High-Resolution Details for Autonomous Drivingcited by 48
  33. 2022StretchBEV: Stretching Future Instance Prediction Spatially and Temporallycited by 47
  34. 2022Uncertainty Quantification of Collaborative Detection for Self-Drivingcited by 46
  35. 20213D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloudcited by 45
  36. 20223D Siamese Transformer Network for Single Object Tracking on Point Cloudscited by 44
  37. 2021Auto4D: Learning to Label 4D Objects from Sequential Point Cloudscited by 42
  38. 2021Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networkscited by 42

50 most cited of 228 All 228 (JSON)