mlsanityData to 14 Oct 2026

NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking

arXiv 2406.15349 ↗First posted 21 Jun 2024cs.CV cs.AI cs.LGcited by 354

On the map: End-to-end autonomous driving → End-to-end system architecture

Abstract (arXiv)

Benchmarking vision-based driving policies is challenging. On one hand, open-loop evaluation with real data is easy, but these results do not reflect closed-loop performance. On the other, closed-loop evaluation is possible in simulation, but is hard to scale due to its significant computational demands. Further, the simulators available today exhibit a large domain gap to real data. This has resulted in an inability to draw clear conclusions from the rapidly growing body of research on end-to-end autonomous driving. In this paper, we present NAVSIM, a middle ground between these evaluation paradigms, where we use large datasets in combination with a non-reactive simulator to enable large-scale real-world benchmarking. Specifically, we gather simulation-based metrics, such as progress and time to collision, by unrolling bird's eye view abstractions of the test scenes for a short simulation horizon. Our simulation is non-reactive, i.e., the evaluated policy and environment do not influence each other. As we demonstrate empirically, this decoupling allows open-loop metric computation while being better aligned with closed-loop evaluations than traditional displacement errors. NAVSIM enabled a new competition held at CVPR 2024, where 143 teams submitted 463 entries, resulting in several new insights. On a large set of challenging scenarios, we observe that simple methods with moderate compute requirements such as TransFuser can match recent large-scale end-to-end driving architectures such as UniAD. Our modular framework can potentially be extended with new datasets, data curation strategies, and metrics, and will be continually maintained to host future challenges. Our code is available at https://github.com/autonomousvision/navsim.

Key facts. Cited by 354 papers in the map (data through 14 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 48 of its 66 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

48 of 66 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous driving7 Mar
  2. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?5 Dec
  3. 2023NeuRAD: Neural Rendering for Autonomous Driving26 Nov
  4. 2023FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Driving2 Aug
  5. 2023End-to-end Autonomous Driving: Challenges and Frontiers29 Jun
  6. 2023Hidden Biases of End-to-End Driving Models13 Jun
  7. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes17 May
  8. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving10 May
  9. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning15 Jul
  10. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline16 Jun
  11. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving31 May
  12. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Driving19 Apr
1 more
  1. 2021End-to-end Interpretable Neural Motion Planner17 Jan

Other references · 35

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  3. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  4. 2022Planning-oriented Autonomous Drivingcited by 1,087
  5. 2016End to End Learning for Self-Driving Carscited by 1,009
  6. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  7. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  8. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 397
  9. 2019Learning by Cheatingcited by 383
  10. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  11. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
  12. 2018Learning to Drive in a Daycited by 273
23 more
  1. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 249
  2. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 242
  3. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 212
  4. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 212
  5. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  6. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  7. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 160
  8. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 153
  9. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  10. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  11. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  12. 2021Learning to drive from a world on railscited by 114
  13. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  14. 2018Baidu Apollo EM Motion Plannercited by 100
  15. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 93
  16. 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
  17. 2018Conditional Affordance Learning for Driving in Urban Environmentscited by 70
  18. 2018On Offline Evaluation of Vision-based Driving Modelscited by 64
  19. 2024Embodied Understanding of Driving Scenarioscited by 63
  20. 2019Jointly Learnable Behavior and Trajectory Planning for Self-Driving Vehiclescited by 51
  21. 2024SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Trafficcited by 27
  22. 2023Coaching a Teachable Studentcited by 19
  23. 2021Learning Interactive Driving Policies via Data-driven Simulationcited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 15; 2025: 131; 2026: 208.

354 papers.

Sort:

By area:

  1. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
  2. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 303
  3. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
  4. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 231
  5. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 198
  6. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 176
  7. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 166
  8. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
  9. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 151
  10. 2025Pseudo-Simulation for Autonomous Drivingcited by 150
  11. 2025Unified Vision-Language-Action Modelcited by 134
  12. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 134
38 more
  1. 2025GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Drivingcited by 120
  2. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 117
  3. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  4. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  5. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Modelcited by 95
  6. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 94
  7. 2024DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mambacited by 87
  8. 2025ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Drivingcited by 67
  9. 2025Generalized Trajectory Scoring for End-to-end Multimodal Planningcited by 67
  10. 20253D and 4D World Modeling: A Surveycited by 67
  11. 2025DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Drivingcited by 64
  12. 2025WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarioscited by 62
  13. 2025A Survey of World Models for Autonomous Drivingcited by 60
  14. 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Modelscited by 60
  15. 2025AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Drivingcited by 58
  16. 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Drivingcited by 56
  17. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 51
  18. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 49
  19. 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving Worldcited by 47
  20. 2026DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generationcited by 45
  21. 2025ReSim: Reliable World Simulation for Autonomous Drivingcited by 44
  22. 2026Driving on Registerscited by 44
  23. 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuningcited by 42
  24. 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 42
  25. 2025iPad: Iterative Proposal-centric End-to-End Autonomous Drivingcited by 42
  26. 2025RAP: 3D Rasterization Augmented End-to-End Planningcited by 41
  27. 2025Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Trainingcited by 39
  28. 2025DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Drivingcited by 39
  29. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 38
  30. 2026SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Drivingcited by 38
  31. 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Drivingcited by 37
  32. 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 36
  33. 2024DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenescited by 35
  34. 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modelingcited by 35
  35. 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
  36. 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 33
  37. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 33
  38. 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Drivingcited by 33

50 most cited of 354 All 354 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving
  2. 2024XLD: A Cross-Lane Dataset for Benchmarking Novel Driving View Synthesis
  3. 2026ReactSim-Bench: Benchmarking Reactive Behavior World Model Simulation in Autonomous Driving
  4. 2024HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving
  5. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles
  6. 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving
  7. 2024Data Scaling Laws for Imitation Learning-Based End-to-End Autonomous Driving
  8. 2025PAVE: An End-to-End Dataset for Production Autonomous Vehicle Evaluation