mlsanityData to 7 Oct 2026

SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representation

arXiv 2405.19620 ↗First posted 30 May 2024cs.CVcited by 239

On the map: End-to-end autonomous driving → End-to-end system architecture

Abstract (arXiv)

The well-established modular autonomous driving system is decoupled into different standalone tasks, e.g. perception, prediction and planning, suffering from information loss and error accumulation across modules. In contrast, end-to-end paradigms unify multi-tasks into a fully differentiable framework, allowing for optimization in a planning-oriented spirit. Despite the great potential of end-to-end paradigms, both the performance and efficiency of existing methods are not satisfactory, particularly in terms of planning safety. We attribute this to the computationally expensive BEV (bird's eye view) features and the straightforward design for prediction and planning. To this end, we explore the sparse representation and review the task design for end-to-end autonomous driving, proposing a new paradigm named SparseDrive. Concretely, SparseDrive consists of a symmetric sparse perception module and a parallel motion planner. The sparse perception module unifies detection, tracking and online mapping with a symmetric model architecture, learning a fully sparse representation of the driving scene. For motion prediction and planning, we review the great similarity between these two tasks, leading to a parallel design for motion planner. Based on this parallel design, which models planning as a multi-modal problem, we propose a hierarchical planning selection strategy , which incorporates a collision-aware rescore module, to select a rational and safe trajectory as the final planning output. With such effective designs, SparseDrive surpasses previous state-of-the-arts by a large margin in performance of all tasks, while achieving much higher training and inference efficiency. Code will be avaliable at https://github.com/swc-17/SparseDrive for facilitating future research.

Key facts. Cited by 239 papers in the map (data through 7 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 50 of its 56 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

50 of 56 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?5 Dec
  2. 2023FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Driving2 Aug
  3. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes17 May
  4. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning15 Jul
  5. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Driving19 Apr

Other references · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  7. 2022Planning-oriented Autonomous Drivingcited by 1,068
  8. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  9. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  10. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  11. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  12. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
33 more
  1. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  2. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 515
  3. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  4. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 379
  5. 2020TransTrack: Multiple Object Tracking with Transformercited by 370
  6. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  7. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  8. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  9. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  10. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
  11. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  12. 2021HDMapNet: An Online HD Map Construction and Evaluation Frameworkcited by 280
  13. 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 243
  14. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  15. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  16. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  17. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  18. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  19. 2022MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectorscited by 148
  20. 2020PnPNet: End-to-End Perception and Prediction with Tracking in the Loopcited by 143
  21. 2023StreamMapNet: Streaming Mapping Network for Vectorized Online HD Map Constructioncited by 143
  22. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  23. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  24. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 92
  25. 2022MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queriescited by 90
  26. 2022Differentiable Raycasting for Self-supervised Occupancy Forecastingcited by 83
  27. 2021Monocular Quasi-Dense 3D Object Trackingcited by 80
  28. 2022Efficient and Robust 2D-to-BEV Representation Learning via Geometry-guided Kernel Transformercited by 80
  29. 2022BEVPoolv2: A Cutting-edge Implementation of BEVDet Toward Deploymentcited by 79
  30. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 69
  31. 2023MOTRv3: Release-Fetch Supervision for End-to-End Multi-Object Trackingcited by 42
  32. 2022Perceive, Interact, Predict: Learning Dynamic and Static Clues for End-to-End Motion Predictioncited by 28
  33. 2024GraphAD: Interaction Scene Graph for End-to-end Autonomous Drivingcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 105; 2026: 118.

239 papers.

Sort:

By area:

  1. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  2. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  3. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 125
  4. 2025GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Drivingcited by 117
  5. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  6. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Modelcited by 91
  7. 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervisioncited by 85
  8. 2025Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Drivingcited by 84
  9. 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learningcited by 76
  10. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  11. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Drivingcited by 67
  12. 2025ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Drivingcited by 66
38 more
  1. 2025DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Drivingcited by 64
  2. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Modelcited by 51
  3. 2024V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Modelscited by 47
  4. 2024Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Drivingcited by 46
  5. 2025iPad: Iterative Proposal-centric End-to-End Autonomous Drivingcited by 41
  6. 2025DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Drivingcited by 39
  7. 2025HiP-AD: Hierarchical and Multi-Granularity Planning with Deformable Attention for Autonomous Driving in a Single Decodercited by 35
  8. 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 34
  9. 2026SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Drivingcited by 34
  10. 2025AutoDrive-R2: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Drivingcited by 32
  11. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  12. 2024GaussianAD: Gaussian-Centric End-to-End Autonomous Drivingcited by 31
  13. 2025IRL-VLA: Training an Vision-Language-Action Policy via Reward World Modelcited by 31
  14. 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 29
  15. 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Drivingcited by 28
  16. 2024End-to-End Autonomous Driving without Costly Modularization and 3D Manual Annotationcited by 27
  17. 2025ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstructioncited by 27
  18. 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Drivingcited by 27
  19. 2025BEVDriver: Leveraging BEV Maps in LLMs for Robust Closed-Loop Drivingcited by 26
  20. 2026Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Drivingcited by 26
  21. 2025ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Drivingcited by 24
  22. 2025ReasonPlan: Unified Scene Prediction and Decision Reasoning for Closed-loop Autonomous Drivingcited by 22
  23. 2025SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Drivingcited by 21
  24. 2025GuideFlow: Constraint-Guided Flow Matching for Planning in End-to-End Autonomous Drivingcited by 21
  25. 2026SparseDriveV2: Scoring is All You Need for End-to-End Autonomous Drivingcited by 21
  26. 2025WorldRFT: Latent World Model Planning with Reinforcement Fine-Tuning for Autonomous Drivingcited by 19
  27. 2026DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Drivingcited by 18
  28. 2025DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Drivingcited by 17
  29. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Drivingcited by 17
  30. 2024MapExpert: Online HD Map Construction with Simple and Efficient Sparse Map Element Expertcited by 16
  31. 2025Challenger: Affordable Adversarial Driving Video Generationcited by 16
  32. 2025PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Drivingcited by 16
  33. 2025ReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Drivingcited by 15
  34. 2025DrivePI: Spatial-aware 4D MLLM for Unified Autonomous Driving Understanding, Perception, Prediction and Planningcited by 15
  35. 2026DriveFuture: Future-Aware Latent World Models for Autonomous Drivingcited by 15
  36. 2024Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Modelcited by 14
  37. 2025InsightDrive: Insight Scene Representation for End-to-End Autonomous Drivingcited by 14
  38. 2025UncAD: Towards Safe End-to-end Autonomous Driving via Online Map Uncertaintycited by 14

50 most cited of 239 All 239 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Driving
  2. 2023End-to-end Autonomous Driving: Challenges and Frontiers
  3. 2026Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms
  4. 2025DiffAD: A Unified Diffusion Modeling Approach for Autonomous Driving
  5. 2026Lagrange: An Open-Vocabulary, Energy-Based Sparse Framework for Generalized End-to-End Driving
  6. 2025Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution
  7. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities
  8. 2023ADriver-I: A General World Model for Autonomous Driving