mlsanityData to 14 Oct 2026

Unraveling the Effects of Synthetic Data on End-to-End Autonomous Driving

arXiv 2503.18108 ↗First posted 23 Mar 2025cs.RO cs.CVcited by 12

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

End-to-end (E2E) autonomous driving (AD) models require diverse, high-quality data to perform well across various driving scenarios. However, collecting large-scale real-world data is expensive and time-consuming, making high-fidelity synthetic data essential for enhancing data diversity and model robustness. Existing driving simulators for synthetic data generation have significant limitations: game-engine-based simulators struggle to produce realistic sensor data, while NeRF-based and diffusion-based methods face efficiency challenges. Additionally, recent simulators designed for closed-loop evaluation provide limited interaction with other vehicles, failing to simulate complex real-world traffic dynamics. To address these issues, we introduce SceneCrafter, a realistic, interactive, and efficient AD simulator based on 3D Gaussian Splatting (3DGS). SceneCrafter not only efficiently generates realistic driving logs across diverse traffic scenarios but also enables robust closed-loop evaluation of end-to-end models. Experimental results demonstrate that SceneCrafter serves as both a reliable evaluation platform and a efficient data generator that significantly improves end-to-end model generalization.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 44 of its 54 references that are in the map. First posted on arXiv 23 Mar 2025.

Builds on

44 of 54 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving2 Dec
  2. 2024Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generation3 Jun
  3. 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
  4. 2024S-NeRF++: Autonomous Driving Simulation via Neural Reconstruction and Generation3 Feb
  5. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers31 Mar
  6. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning26 Sep
  7. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun
  8. 2017CARLA: An Open Urban Driving Simulator10 Nov

Other references · 36

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2022Planning-oriented Autonomous Drivingcited by 1,087
  5. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  6. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  7. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
  8. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 297
  9. 2018Learning to Drive in a Daycited by 273
  10. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  11. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  12. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 212
24 more
  1. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
  2. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  3. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
  4. 2022Learning from All Vehiclescited by 135
  5. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  6. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  7. 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
  8. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  9. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 96
  10. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 93
  11. 2024NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Drivingcited by 62
  12. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 60
  13. 2023ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modelingcited by 60
  14. 2021Vehicle trajectory prediction works, but not everywherecited by 39
  15. 20243DRealCar: An In-the-wild RGB-D Car Dataset with 360-degree Viewscited by 39
  16. 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversariescited by 38
  17. 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
  18. 2024Are NeRFs ready for autonomous driving? Towards closing the real-to-simulation gapcited by 20
  19. 2024Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End Systemcited by 20
  20. 2021What data do we need for training an AV motion planner?cited by 15
  21. 20233D Data Augmentation for Driving Scenes on Cameracited by 12
  22. 2024SUMO: Search-Based Uncertainty Estimation for Model-Based Offline Reinforcement Learningcited by 12
  23. 2024OASim: an Open and Adaptive Simulator based on Neural Rendering for Autonomous Drivingcited by 10
  24. 2024SynDiff-AD: Improving Semantic Segmentation and End-to-End Autonomous Driving with Synthetic Data from Latent Diffusion Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 7.

12 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving
  2. 2024Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model
  3. 2025RealEngine: Simulating Autonomous Driving in Realistic Context
  4. 2026Real2Sim: A Physics-driven and Editable Gaussian Splatting Framework for Autonomous Driving Scenes
  5. 2025SceneCrafter: Controllable Multi-View Driving Scene Editing
  6. 2026Cam2Sim: Neural Scenario Reconstruction for Closed-Loop Autonomous Driving Simulation
  7. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation
  8. 2026Scaling Self-Play for End-to-End Driving