mlsanityData to 7 Oct 2026

Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Driving

arXiv 2409.18341 ↗First posted 26 Sep 2024cs.CVcited by 46

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

End-to-End Autonomous Driving (E2EAD) methods typically rely on supervised perception tasks to extract explicit scene information (e.g., objects, maps). This reliance necessitates expensive annotations and constrains deployment and data scalability in real-time applications. In this paper, we introduce SSR, a novel framework that utilizes only 16 navigation-guided tokens as Sparse Scene Representation, efficiently extracting crucial scene information for E2EAD. Our method eliminates the need for human-designed supervised sub-tasks, allowing computational resources to concentrate on essential elements directly related to navigation intent. We further introduce a temporal enhancement module, aligning predicted future scenes with actual future scenes through self-supervision. SSR achieves a 27.2% relative reduction in L2 error and a 51.6% decrease in collision rate to UniAD in nuScenes, with a 10.9× faster inference speed and 13× faster training time. Moreover, SSR outperforms VAD-Base with a 48.6-point improvement on driving score in CARLA's Town05 Long benchmark. This framework represents a significant leap in real-time autonomous driving systems and paves the way for future scalable deployment. Code is available at https://github.com/PeidongLi/SSR.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 38 of its 43 references that are in the map. First posted on arXiv 26 Sep 2024.

Builds on

38 of 43 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
  2. 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
  3. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving27 Nov
  4. 2022Differentiable Raycasting for Self-supervised Occupancy Forecasting4 Oct
  5. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers31 Mar
  6. 2017CARLA: An Open Urban Driving Simulator10 Nov

Other references · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2017Squeeze-and-Excitation Networkscited by 4,763
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2022Planning-oriented Autonomous Drivingcited by 1,068
  6. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  7. 2016End to End Learning for Self-Driving Carscited by 1,000
  8. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  9. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  10. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  11. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  12. 2019Learning by Cheatingcited by 361
20 more
  1. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  2. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  3. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
  4. 2021HDMapNet: An Online HD Map Construction and Evaluation Frameworkcited by 280
  5. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
  6. 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 243
  7. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  8. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  9. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 239
  10. 2023Scene as Occupancycited by 208
  11. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  12. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  13. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  14. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 149
  15. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 142
  16. 2023Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Predictioncited by 49
  17. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 46
  18. 2024End-to-End Autonomous Driving without Costly Modularization and 3D Manual Annotationcited by 27
  19. 2024DualBEV: Unifying Dual View Transformation with Probabilistic Correspondencescited by 9
  20. 2024MIM4D: Masked Modeling with Multi-View Video for Autonomous Driving Representation Learningcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 25; 2026: 21.

46 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  2. 20253D and 4D World Modeling: A Surveycited by 64
  3. 2025A Survey of World Models for Autonomous Drivingcited by 60
  4. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  5. 2025DriveX: Omni Scene Modeling for Learning Generalizable World Knowledge in Autonomous Drivingcited by 20
  6. 2025PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Drivingcited by 16
  7. 2025ReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Drivingcited by 15
  8. 2026ResWorld: Temporal Residual World Model for End-to-End Autonomous Drivingcited by 14
  9. 2026DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Drivingcited by 11
  10. 2025RoCA: Robust Cross-Domain End-to-End Autonomous Drivingcited by 8
  11. 2025TrajDiff: End-to-end Autonomous Driving without Perception Annotationcited by 8
  12. 2026SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse Worldcited by 7
34 more
  1. 2025HENet++: Hybrid Encoding and Multi-task Learning for 3D Perception and End-to-end Autonomous Drivingcited by 6
  2. 2026GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Drivingcited by 6
  3. 2025VR-Drive: Viewpoint-Robust End-to-End Driving with Feed-Forward 3D Gaussian Splattingcited by 5
  4. 2025FlowDrive: Energy Flow Field for End-to-End Autonomous Drivingcited by 4
  5. 2025WPT: World-to-Policy Transfer via Online World Model Distillationcited by 4
  6. 2025KnowVal: A Knowledge-Augmented and Value-Guided Autonomous Driving Systemcited by 4
  7. 2026FlowAD: Ego-Scene Interactive Modeling for Autonomous Drivingcited by 4
  8. 2025Echo Planning for Autonomous Driving: From Current Observations to Future Trajectories and Backcited by 3
  9. 2025Vision-Centric 4D Occupancy Forecasting and Planning via Implicit Residual World Modelscited by 3
  10. 2026Zero-Shot Cross-City Generalization in End-to-End Autonomous Driving: Self-Supervised versus Supervised Representationscited by 3
  11. 2025Restore Text First, Enhance Image Later: Two-Stage Scene Text Image Super-Resolution with Glyph Structure Guidancecited by 2
  12. 2025Embodied Cognition Augmented End2End Autonomous Drivingcited by 2
  13. 2025LAP: Fast LAtent Diffusion Planner for Autonomous Drivingcited by 2
  14. 2026UniUncer: Unified Dynamic Static Uncertainty for End to End Drivingcited by 2
  15. 2026DynFlowDrive: Flow-Based Dynamic World Modeling for Autonomous Drivingcited by 2
  16. 2026DLWM: Dual Latent World Models enable Holistic Gaussian-centric Pre-training in Autonomous Drivingcited by 2
  17. 2026HEAT: Heterogeneous End-to-End Autonomous Driving via Trajectory-Guided World Modelscited by 2
  18. 2026IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Drivingcited by 2
  19. 2026CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Drivingcited by 2
  20. 2025Closing the Navigation Compliance Gap in End-to-end Autonomous Drivingcited by 1
  21. 2026DeepSight: Long-Horizon World Modeling via Latent States Prediction for End-to-End Autonomous Drivingcited by 1
  22. 2026The Road Ahead in Autonomous Driving: The KITScenes Multimodal Datasetcited by 1
  23. 2025Less is More: Lean yet Powerful Vision-Language Model for Autonomous Drivingcited by 0
  24. 2025CoIRL-AD: Collaborative-Competitive Imitation-Reinforcement Learning in Latent World Models for Autonomous Drivingcited by 0
  25. 2025Constructing the Umwelt: Cognitive Planning through Belief-Intent Co-Evolutioncited by 0
  26. 2025Dataset Safety in Autonomous Driving: Requirements, Risks, and Assurancecited by 0
  27. 2025ISS Policy : Scalable Diffusion Policy with Implicit Scene Supervisioncited by 0
  28. 2026Temporal Sampling Frequency Matters: A Capacity-Aware Study of End-to-End Driving Trajectory Predictioncited by 0
  29. 2026NudgeVAD: Language-Nudged End-to-End Driving via FiLM Residualscited by 0
  30. 2026PLAN-S: Bridging Planning with Latent Style Dynamics for Autonomous Driving World Modelscited by 0
  31. 2026GraphBEV++: Multi-Modal Feature Alignment for Autonomous Drivingcited by 0
  32. 2026BeyondSight: Object Permanence for End-to-End Autonomous Drivingcited by 0
  33. 2026Plug-and-Play Traffic Element Awareness for End-to-End Autonomous Drivingcited by 0
  34. 2026AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SparseWorld: Enhancing End-to-End Autonomous Driving via World Models with Sparse Scene Representation
  2. 2025Towards Efficient and Effective Multi-Camera Encoding for End-to-End Driving
  3. 2025InsightDrive: Insight Scene Representation for End-to-End Autonomous Driving
  4. 2026NTR: Neural Token Reconstruction for Scene Token Bottleneck in End-to-End Driving
  5. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving
  6. 2024DualAD: Disentangling the Dynamic and Static World for End-to-End Driving
  7. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model
  8. 2025NavigScene: Bridging Local Perception and Global Navigation for Beyond-Visual-Range Autonomous Driving