mlsanityData to 7 Oct 2026

Pseudo-Simulation for Autonomous Driving

arXiv 2506.04218 ↗First posted 4 Jun 2025cs.RO cs.AI cs.CVcited by 141

On the map: End-to-end autonomous driving → Adversarial scenarios and safety testing

Abstract (arXiv)

Existing evaluation paradigms for Autonomous Vehicles (AVs) face critical limitations. Real-world evaluation is often challenging due to safety concerns and a lack of reproducibility, whereas closed-loop simulation can face insufficient realism or high computational costs. Open-loop evaluation, while being efficient and data-driven, relies on metrics that generally overlook compounding errors. In this paper, we propose pseudo-simulation, a novel paradigm that addresses these limitations. Pseudo-simulation operates on real datasets, similar to open-loop evaluation, but augments them with synthetic observations generated prior to evaluation using 3D Gaussian Splatting. Our key idea is to approximate potential future states the AV might encounter by generating a diverse set of observations that vary in position, heading, and speed. Our method then assigns a higher importance to synthetic observations that best match the AV's likely behavior using a novel proximity-based weighting scheme. This enables evaluating error recovery and the mitigation of causal confusion, as in closed-loop benchmarks, without requiring sequential interactive simulation. We show that pseudo-simulation is better correlated with closed-loop simulations (R2=0.8) than the best existing open-loop approach (R2=0.7). We also establish a public leaderboard for the community to benchmark new methodologies with pseudo-simulation. Our code is available at https://github.com/autonomousvision/navsim.

Key facts. Cited by 141 papers in the map (data through 7 Oct 2026). Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving. Builds on 56 of its 73 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

56 of 73 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Driving11 Apr
  2. 2024Are NeRFs ready for autonomous driving? Towards closing the real-to-simulation gap24 Mar

Other references · 54

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  6. 2016End to End Learning for Self-Driving Carscited by 1,000
  7. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  8. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  9. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  10. 2019Learning by Cheatingcited by 361
  11. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  12. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
38 more
  1. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  2. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  3. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  4. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  5. 2023GS-SLAM: Dense Visual SLAM with 3D Gaussian Splattingcited by 278
  6. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 244
  7. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  8. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  9. 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splattingcited by 218
  10. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
  11. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 160
  12. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
  13. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  14. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  15. 2023NeuRAD: Neural Rendering for Autonomous Drivingcited by 146
  16. 2022KISS-ICP: In Defense of Point-to-Point ICP -- Simple, Accurate, and Robust Registration If Done the Right Waycited by 132
  17. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  18. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  19. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  20. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  21. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
  22. 2023Hidden Biases of End-to-End Driving Modelscited by 95
  23. 2020Counterfactual Data Augmentation using Locally Factored Dynamicscited by 93
  24. 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 93
  25. 2024HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Drivingcited by 84
  26. 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
  27. 2020Fighting Copycat Agents in Behavioral Cloning from Observation Historiescited by 70
  28. 2018On Offline Evaluation of Vision-based Driving Modelscited by 62
  29. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
  30. 2024Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generationcited by 54
  31. 2024STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenescited by 53
  32. 2022MoCoDA: Model-based Counterfactual Data Augmentationcited by 44
  33. 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPScited by 43
  34. 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
  35. 2020Label Efficient Visual Abstractions for Autonomous Drivingcited by 33
  36. 2024Hidden Biases of End-to-End Driving Datasetscited by 30
  37. 2024SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Trafficcited by 27
  38. 2025EVolSplat: Efficient Volume-based Gaussian Splatting for Urban View Synthesiscited by 20

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 30; 2026: 111.

141 papers.

Sort:

By area:

  1. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 127
  2. 20253D and 4D World Modeling: A Surveycited by 64
  3. 2025Generalized Trajectory Scoring for End-to-end Multimodal Planningcited by 63
  4. 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Drivingcited by 52
  5. 2025RAP: 3D Rasterization Augmented End-to-End Planningcited by 39
  6. 2026Driving on Registerscited by 38
  7. 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 34
  8. 2025IRL-VLA: Training an Vision-Language-Action Policy via Reward World Modelcited by 31
  9. 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Drivingcited by 31
  10. 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Drivingcited by 27
  11. 2026Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Drivingcited by 26
  12. 2026LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Drivingcited by 26
38 more
  1. 2025LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Drivingcited by 22
  2. 2025Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Drivingcited by 19
  3. 2025DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Drivingcited by 17
  4. 2026Unleashing VLA Potentials in Autonomous Driving via Explicit Learning from Failurescited by 17
  5. 2026DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planningcited by 17
  6. 2025PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Drivingcited by 16
  7. 2025Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorercited by 16
  8. 2025WAM-Flow: Parallel Coarse-to-Fine Motion Planning via Discrete Flow Matching for Autonomous Drivingcited by 15
  9. 2026SpanVLA: Efficient Action Bridging and Learning from Negative-Recovery Samples for Vision-Language-Action Modelcited by 15
  10. 2026DriveFuture: Future-Aware Latent World Models for Autonomous Drivingcited by 15
  11. 2026Devil is in Narrow Policy: Unleashing Exploration in Driving VLA Modelscited by 14
  12. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representationcited by 13
  13. 2026AutoDrive-P3: Unified Chain of Perception-Prediction-Planning Thought via Reinforcement Fine-Tuningcited by 13
  14. 2025ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Drivingcited by 12
  15. 2025MindDrive: An All-in-One Framework Bridging World Models and Vision-Language Model for End-to-End Autonomous Drivingcited by 12
  16. 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scalecited by 12
  17. 2026EponaV2: Driving World Model with Comprehensive Future Reasoningcited by 12
  18. 2026MeanFuser: Fast One-Step Multi-Modal Trajectory Generation and Adaptive Reconstruction via MeanFlow for End-to-End Autonomous Drivingcited by 11
  19. 2025WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Drivingcited by 10
  20. 2026ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Drivingcited by 10
  21. 2026Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigationcited by 10
  22. 2026FLARE: Learning Future-Aware Latent Representations from Vision-Language Models for Autonomous Drivingcited by 9
  23. 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Drivingcited by 9
  24. 2026BridgeSim: Unveiling the OL-CL Gap in End-to-End Autonomous Drivingcited by 9
  25. 2026CLOVER: Closed-Loop Value Estimation and Ranking for End-to-End Autonomous Driving Planningcited by 9
  26. 2025UniSplat: Unified Spatio-Temporal Fusion via 3D Latent Scaffolds for Dynamic Driving Scene Reconstructioncited by 8
  27. 2026Learning from Mistakes: Post-Training for Driving VLA with Takeover Datacited by 8
  28. 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Drivingcited by 8
  29. 2026Test-Time Trajectory Optimization for Autonomous Drivingcited by 7
  30. 2025RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policiescited by 6
  31. 2026Learning to Drive in New Cities Without Human Demonstrationscited by 6
  32. 2026Vega: Learning to Drive with Natural Language Instructionscited by 6
  33. 2026Beyond Imitation: Learning Safe End-to-End Autonomous Driving from Hard Negativescited by 6
  34. 2026GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Drivingcited by 6
  35. 2025DriveE2E: Closed-Loop Benchmark for End-to-End Autonomous Driving through Real-to-Simulationcited by 5
  36. 2026SimWAM: A Simple World Action Model for End-to-End Autonomous Drivingcited by 5
  37. 2026Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planningcited by 4
  38. 2026Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Drivingcited by 4

50 most cited of 141 All 141 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Unraveling the Effects of Synthetic Data on End-to-End Autonomous Driving
  2. 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving
  3. 2025A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systems
  4. 2026Cam2Sim: Neural Scenario Reconstruction for Closed-Loop Autonomous Driving Simulation
  5. 2024XLD: A Cross-Lane Dataset for Benchmarking Novel Driving View Synthesis
  6. 2023SAFR-AV: Safety Analysis of Autonomous Vehicles using Real World Data -- An end-to-end solution for real world data driven scenario-based testing for pre-certification of AV stacks
  7. 2025MultiDrive: A Co-Simulation Framework Bridging 2D and 3D Driving Simulation for AV Software Validation
  8. 2026OVPD: A Virtual-Physical Fusion Testing Dataset of OnSite Auton-omous Driving Challenge