mlsanityData to 7 Oct 2026

VAD: Vectorized Scene Representation for Efficient Autonomous Driving

arXiv 2303.12077 ↗First posted 21 Mar 2023cs.RO cs.CVcited by 613

On the map: End-to-end autonomous driving → VLMs for autonomous driving

Abstract (arXiv)

Autonomous driving requires a comprehensive understanding of the surrounding environment for reliable trajectory planning. Previous works rely on dense rasterized scene representation (e.g., agent occupancy and semantic map) to perform planning, which is computationally intensive and misses the instance-level structure information. In this paper, we propose VAD, an end-to-end vectorized paradigm for autonomous driving, which models the driving scene as a fully vectorized representation. The proposed vectorized paradigm has two significant advantages. On one hand, VAD exploits the vectorized agent motion and map elements as explicit instance-level planning constraints which effectively improves planning safety. On the other hand, VAD runs much faster than previous end-to-end planning methods by getting rid of computation-intensive rasterized representation and hand-designed post-processing steps. VAD achieves state-of-the-art end-to-end planning performance on the nuScenes dataset, outperforming the previous best method by a large margin. Our base model, VAD-Base, greatly reduces the average collision rate by 29.0% and runs 2.5x faster. Besides, a lightweight variant, VAD-Tiny, greatly improves the inference speed (up to 9.3x) while achieving comparable planning performance. We believe the excellent performance and the high efficiency of VAD are critical for the real-world deployment of an autonomous driving system. Code and models are available at https://github.com/hustvl/VAD for facilitating future research.

Key facts. Cited by 613 papers in the map (data through 7 Oct 2026). Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving. Builds on 43 of its 52 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

43 of 52 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  6. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  7. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  8. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  9. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  10. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  11. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  12. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
31 more
  1. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  2. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  3. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  4. 2019Learning by Cheatingcited by 361
  5. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  6. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
  7. 2021HDMapNet: An Online HD Map Construction and Evaluation Frameworkcited by 280
  8. 2018Learning to Drive in a Daycited by 266
  9. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  10. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  11. 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 243
  12. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  13. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  14. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  15. 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
  16. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  17. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  18. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  19. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 147
  20. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  21. 2021Learning to drive from a world on railscited by 114
  22. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  23. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  24. 2022Differentiable Raycasting for Self-supervised Occupancy Forecastingcited by 83
  25. 2022Efficient and Robust 2D-to-BEV Representation Learning via Geometry-guided Kernel Transformercited by 80
  26. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 59
  27. 2023Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Constructioncited by 50
  28. 2022Perceive, Interact, Predict: Learning Dynamic and Static Clues for End-to-End Motion Predictioncited by 28
  29. 2022Vision-based Uneven BEV Representation Learning with Polar Rasterization and Surface Estimationcited by 23
  30. 2021What data do we need for training an AV motion planner?cited by 15
  31. 2022HOPE: Hierarchical Spatial-temporal Network for Occupancy Flow Predictioncited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 25; 2024: 97; 2025: 250; 2026: 241.

613 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  3. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  4. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  5. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  6. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  7. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  8. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  9. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 258
  10. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  11. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
  12. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 244
38 more
  1. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  2. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 239
  3. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  4. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
  5. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  6. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  7. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 192
  8. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
  9. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  10. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
  11. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 172
  12. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 165
  13. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
  14. 2023A Language Agent for Autonomous Drivingcited by 161
  15. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
  16. 2024Behavior Generation with Latent Actionscited by 159
  17. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
  18. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
  19. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  20. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 145
  21. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 142
  22. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 136
  23. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  24. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 127
  25. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
  26. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 125
  27. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 124
  28. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  29. 2025GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Drivingcited by 117
  30. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 115
  31. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
  32. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Drivingcited by 105
  33. 2023ADriver-I: A General World Model for Autonomous Drivingcited by 102
  34. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  35. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  36. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  37. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  38. 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 93

50 most cited of 613 All 613 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024GPD-1: Generative Pre-training for Driving
  2. 2025LAP: Fast LAtent Diffusion Planner for Autonomous Driving
  3. 2025MTR-VP: Towards End-to-End Trajectory Planning through Context-Driven Image Encoding and Multiple Trajectory Prediction
  4. 2024QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Driving
  5. 2025ViLaD: A Large Vision Language Diffusion Framework for End-to-End Autonomous Driving
  6. 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model
  7. 2026Do Better Visual Representations Always Lead to Better End-to-End Autonomous Driving?
  8. 2025VDT-Auto: End-to-end Autonomous Driving with VLM-Guided Diffusion Transformers