mlsanityData to 7 Oct 2026

Synthetic Datasets for Autonomous Driving: A Survey

arXiv 2304.12205 ↗First posted 24 Apr 2023eess.IV cs.CVcited by 48

Not on the map. Why?

Abstract (arXiv)

Autonomous driving techniques have been flourishing in recent years while thirsting for huge amounts of high-quality data. However, it is difficult for real-world datasets to keep up with the pace of changing requirements due to their expensive and time-consuming experimental and labeling costs. Therefore, more and more researchers are turning to synthetic datasets to easily generate rich and changeable data as an effective complement to the real world and to improve the performance of algorithms. In this paper, we summarize the evolution of synthetic dataset generation methods and review the work to date in synthetic datasets related to single and multi-task categories for to autonomous driving study. We also discuss the role that synthetic dataset plays the evaluation, gap test, and positive effect in autonomous driving related algorithm testing, especially on trustworthiness and safety aspects. Finally, we discuss general trends and possible development directions. To the best of our knowledge, this is the first survey focusing on the application of synthetic datasets in autonomous driving. This survey also raises awareness of the problems of real-world deployment of autonomous driving technology and provides researchers with a possible solution.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 113 of its 174 references that are in the map. First posted on arXiv 24 Apr 2023.

Builds on

113 of 174 references.

References · 113

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  9. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  10. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  11. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  12. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
38 more
  1. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  2. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  3. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  4. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
  5. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  6. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  7. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  8. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  9. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  10. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  11. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  12. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  13. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  14. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  15. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  16. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  17. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  18. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
  19. 2020Long-tail learning via logit adjustmentcited by 605
  20. 2018Learning to Adapt Structured Output Space for Semantic Segmentationcited by 584
  21. 2016Direct Sparse Odometrycited by 479
  22. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  23. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 411
  24. 2017Semantic Foggy Scene Understanding with Synthetic Datacited by 401
  25. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 388
  26. 2020Virtual KITTI 2cited by 345
  27. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 340
  28. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  29. 2019Bidirectional Learning for Domain Adaptation of Semantic Segmentationcited by 308
  30. 2021OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communicationcited by 304
  31. 2018Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomizationcited by 292
  32. 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 279
  33. 2017Data Augmentation Generative Adversarial Networkscited by 278
  34. 2022DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 270
  35. 2018Strong-Weak Distribution Alignment for Adaptive Object Detectioncited by 269
  36. 2022Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Mapscited by 255
  37. 2019SEAN: Image Synthesis with Semantic Region-Adaptive Normalizationcited by 253
  38. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223

50 most cited of 113 All 113 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 8; 2025: 19; 2026: 16.

48 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 65
  3. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  4. 2023Deep Transfer Learning for Intelligent Vehicle Perception: a Surveycited by 10
  5. 2025GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generationcited by 10
  6. 2025Collaborative Perception Datasets for Autonomous Driving: A Reviewcited by 8
  7. 2026ACCIDENT: A Benchmark Dataset for Vehicle Accident Detection from Traffic Surveillance Videoscited by 6
  8. 2024PC-NeRF: Parent-Child Neural Radiance Fields Using Sparse LiDAR Frames in Autonomous Driving Environmentscited by 5
  9. 2024Vulnerable Road User Detection and Safety Enhancement: A Comprehensive Surveycited by 5
  10. 2024Hard Cases Detection in Motion Prediction by Vision-Language Foundation Modelscited by 4
  11. 2025AgentSGEN: Multi-Agent LLM in the Loop for Semantic Collaboration and GENeration of Synthetic Datacited by 4
  12. 2023PC-NeRF: Parent-Child Neural Radiance Fields under Partial Sensor Data Loss in Autonomous Driving Environmentscited by 3
36 more
  1. 2024TK-Planes: Tiered K-Planes with High Dimensional Feature Vectors for Dynamic UAV-based Scenescited by 3
  2. 2024IMFL-AIGC: Incentive Mechanism Design for Federated Learning Empowered by Artificial Intelligence Generated Contentcited by 3
  3. 2024SynDiff-AD: Improving Semantic Segmentation and End-to-End Autonomous Driving with Synthetic Data from Latent Diffusion Modelscited by 3
  4. 2025SimWorld: A Unified Benchmark for Simulator-Conditioned Scene Generation via World Modelcited by 3
  5. 2025Large Language Models for Crash Detection in Video: A Survey of Methods, Datasets, and Challengescited by 3
  6. 2025Scaling Up Forest Vision with Synthetic Datacited by 3
  7. 2025Synthetic Dataset Evaluation Based on Generalized Cross Validationcited by 3
  8. 2025LangDriveCTRL: Natural Language Controllable Driving Scene Editing with Multi-modal Agentscited by 3
  9. 2026A Sustainable AI Economy Needs Data Deals That Work for Generatorscited by 3
  10. 2025A Style-Based Profiling Framework for Quantifying the Synthetic-to-Real Gap in Autonomous Driving Datasetscited by 2
  11. 2025DetectiumFire: A Comprehensive Multi-modal Dataset Bridging Vision and Language for Fire Understandingcited by 2
  12. 2025SimBEV: A Synthetic Multi-Task Multi-Sensor Driving Data Generation Tool and Datasetcited by 1
  13. 2025Evaluating the Impact of Synthetic Data on Object Detection Tasks in Autonomous Drivingcited by 1
  14. 2025Synthetic Dataset Generation for Autonomous Mobile Robots Using 3D Gaussian Splatting for Vision Trainingcited by 1
  15. 2025David vs. Goliath: A comparative study of different-sized LLMs for code generation in the domain of automotive scenario generationcited by 1
  16. 2026On the Assessment of Sensitivity of Autonomous Vehicle Perceptioncited by 1
  17. 2023Utilizing dataset affinity prediction in object detection to assess training datacited by 0
  18. 2023A Unified Generative Framework for Realistic Lidar Simulation in Autonomous Driving Systemscited by 0
  19. 2024EucliDreamer: Fast and High-Quality Texturing for 3D Models with Depth-Conditioned Stable Diffusioncited by 0
  20. 2025AutoGeTS: Knowledge-based Automated Generation of Text Synthetics for Improving Text Classificationcited by 0
  21. 2025Canonical Space Representation for 4D Panoptic Segmentation of Articulated Objectscited by 0
  22. 2025KPLM-STA: Physically-Accurate Shadow Synthesis for Human Relighting via Keypoint-Based Light Modelingcited by 0
  23. 2025Dataset Safety in Autonomous Driving: Requirements, Risks, and Assurancecited by 0
  24. 2026Beyond Cropping and Rotation: Automated Evolution of Powerful Task-Specific Augmentations with Generative Modelscited by 0
  25. 2026Motion Capture is Not the Target Domain: Scaling Synthetic Data for Learning Motion Representationscited by 0
  26. 2026Developing AI Agents with Simulated Data: Why, what, and how?cited by 0
  27. 2026Scalable Evaluation of the Realism of Synthetic Environmental Augmentations in Imagescited by 0
  28. 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Drivingcited by 0
  29. 2026Human-Inspired Context-Selective Multimodal Memory for Social Robotscited by 0
  30. 2026From Concept to Capability: Evaluating 3D Gaussian Splatting for Synthetic Scene Editing in Autonomous Drivingcited by 0
  31. 2026Generative Texture Diversification of 3D Pedestrians for Robust Autonomous Driving Perceptioncited by 0
  32. 2026CityGen: Structure-Guided City-Style Synthesis for Cross-City Autonomous Drivingcited by 0
  33. 2026ASTAD: Asymmetric Style Transfer for Synthetic-to-Real Adaptation in Autonomous Drivingcited by 0
  34. 2026presto: Efficient, Training-free, and Open-world Object Placement via Imaginary Searchcited by 0
  35. 2026Curating Synthetic Data for Task-Specific Visual Perceptioncited by 0
  36. 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platformcited by 0