Unraveling the Effects of Synthetic Data on End-to-End Autonomous Driving
On the map: World models → World models for autonomous driving
also in 1 more
Abstract (arXiv)
End-to-end (E2E) autonomous driving (AD) models require diverse, high-quality data to perform well across various driving scenarios. However, collecting large-scale real-world data is expensive and time-consuming, making high-fidelity synthetic data essential for enhancing data diversity and model robustness. Existing driving simulators for synthetic data generation have significant limitations: game-engine-based simulators struggle to produce realistic sensor data, while NeRF-based and diffusion-based methods face efficiency challenges. Additionally, recent simulators designed for closed-loop evaluation provide limited interaction with other vehicles, failing to simulate complex real-world traffic dynamics. To address these issues, we introduce SceneCrafter, a realistic, interactive, and efficient AD simulator based on 3D Gaussian Splatting (3DGS). SceneCrafter not only efficiently generates realistic driving logs across diverse traffic scenarios but also enables robust closed-loop evaluation of end-to-end models. Experimental results demonstrate that SceneCrafter serves as both a reliable evaluation platform and a efficient data generator that significantly improves end-to-end model generalization.
Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 44 of its 54 references that are in the map. First posted on arXiv 23 Mar 2025.
Builds on
44 of 54 references.
Closest ancestors · 8
Same direction, newest first
- 2024HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving2 Dec
- 2024Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generation3 Jun
- 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
- 2024S-NeRF++: Autonomous Driving Simulation via Neural Reconstruction and Generation3 Feb
- 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers31 Mar
- 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning26 Sep
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun
- 2017CARLA: An Open Urban Driving Simulator10 Nov
Other references · 36
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
- 2022Planning-oriented Autonomous Drivingcited by 1,087
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
- 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
- 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 297
- 2018Learning to Drive in a Daycited by 273
- 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
- 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
- 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 212
24 more
- 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
- 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
- 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
- 2022Learning from All Vehiclescited by 135
- 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
- 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
- 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
- 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
- 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 96
- 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 93
- 2024NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Drivingcited by 62
- 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 60
- 2023ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modelingcited by 60
- 2021Vehicle trajectory prediction works, but not everywherecited by 39
- 20243DRealCar: An In-the-wild RGB-D Car Dataset with 360-degree Viewscited by 39
- 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversariescited by 38
- 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
- 2024Are NeRFs ready for autonomous driving? Towards closing the real-to-simulation gapcited by 20
- 2024Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End Systemcited by 20
- 2021What data do we need for training an AV motion planner?cited by 15
- 20233D Data Augmentation for Driving Scenes on Cameracited by 12
- 2024SUMO: Search-Based Uncertainty Estimation for Model-Based Offline Reinforcement Learningcited by 12
- 2024OASim: an Open and Adaptive Simulator based on Neural Rendering for Autonomous Drivingcited by 10
- 2024SynDiff-AD: Improving Semantic Segmentation and End-to-End Autonomous Driving with Synthetic Data from Latent Diffusion Modelscited by 3
Built on by
Citing papers per year: 2025: 5; 2026: 7.
12 papers.
- 20253D and 4D World Modeling: A Surveycited by 67
- 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 33
- 2025X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllabilitycited by 20
- 2025Model-Based Policy Adaptation for Closed-Loop End-to-End Autonomous Drivingcited by 7
- 2026From Articulated Kinematics to Routed Visual Control for Action-Conditioned Surgical Video Generationcited by 4
- 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challengescited by 3
- 2026FrozenDrive: Zero-Shot Text-Guided Driving Scene Generation and Data Augmentation with Parameter-Free Frozen Diffusion Modelcited by 1
- 2025ToosiCubix: Monocular 3D Cuboid Labeling via Vehicle Part Annotationscited by 0
- 2026Closed Loop Dynamic Driving Data Mixture for Real-Synthetic Co-Trainingcited by 0
- 2026DecoupleGS: Interactive 3D Gaussian Splatting for End-to-End Autonomous Driving Testingcited by 0
- 2026GeoFlow: Efficient Driving Video Generation via Geometry-Aligned Priorscited by 0
- 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulationcited by 0
- 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulation2 Sep 2026
- 2026GeoFlow: Efficient Driving Video Generation via Geometry-Aligned Priors12 Aug 2026
- 2026DecoupleGS: Interactive 3D Gaussian Splatting for End-to-End Autonomous Driving Testing3 Aug 2026
- 2026FrozenDrive: Zero-Shot Text-Guided Driving Scene Generation and Data Augmentation with Parameter-Free Frozen Diffusion Model18 Jun 2026
- 2026Closed Loop Dynamic Driving Data Mixture for Real-Synthetic Co-Training20 May 2026
- 2026From Articulated Kinematics to Routed Visual Control for Action-Conditioned Surgical Video Generation9 May 2026
- 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges10 Mar 2026
- 2025Model-Based Policy Adaptation for Closed-Loop End-to-End Autonomous Driving26 Nov 2025
- 2025OmniNWM: Omniscient Driving Navigation World Models21 Oct 2025
- 20253D and 4D World Modeling: A Survey4 Sep 2025
- 2025ToosiCubix: Monocular 3D Cuboid Labeling via Vehicle Part Annotations26 Jun 2025
- 2025X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllability16 Jun 2025
Similar, not cited
Close in wording, no citation either way
- 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving
- 2024Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model
- 2025RealEngine: Simulating Autonomous Driving in Realistic Context
- 2026Real2Sim: A Physics-driven and Editable Gaussian Splatting Framework for Autonomous Driving Scenes
- 2025SceneCrafter: Controllable Multi-View Driving Scene Editing
- 2026Cam2Sim: Neural Scenario Reconstruction for Closed-Loop Autonomous Driving Simulation
- 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation
- 2026Scaling Self-Play for End-to-End Driving