mlsanityData to 7 Oct 2026

How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence

arXiv 2305.01263 ↗First posted 2 May 2023cs.RO cs.AIcited by 55

Not on the map. Why?

Abstract (arXiv)

Safety and cost are two important concerns for the development of autonomous driving technologies. From the academic research to commercial applications of autonomous driving vehicles, sufficient simulation and real world testing are required. In general, a large scale of testing in simulation environment is conducted and then the learned driving knowledge is transferred to the real world, so how to adapt driving knowledge learned in simulation to reality becomes a critical issue. However, the virtual simulation world differs from the real world in many aspects such as lighting, textures, vehicle dynamics, and agents' behaviors, etc., which makes it difficult to bridge the gap between the virtual and real worlds. This gap is commonly referred to as the reality gap (RG). In recent years, researchers have explored various approaches to address the reality gap issue, which can be broadly classified into three categories: transferring knowledge from simulation to reality (sim2real), learning in digital twins (DTs), and learning by parallel intelligence (PI) technologies. In this paper, we consider the solutions through the sim2real, DTs, and PI technologies, and review important applications and innovations in the field of autonomous driving. Meanwhile, we show the state-of-the-arts from the views of algorithms, models, and simulators, and elaborate the development process from sim2real to DTs and PI. The presentation also illustrates the far-reaching effects and challenges in the development of sim2real, DTs, and PI in autonomous driving.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 42 of its 166 references that are in the map. First posted on arXiv 2 May 2023.

Builds on

42 of 166 references.

References · 42

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  8. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  9. 2016Learning to learn by gradient descent by gradient descentcited by 616
  10. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  11. 2017Robust Adversarial Reinforcement Learningcited by 439
  12. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
30 more
  1. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  2. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  3. 2019Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Datacited by 245
  4. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  5. 2017Teacher-Student Curriculum Learningcited by 197
  6. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  7. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 129
  8. 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
  9. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126
  10. 2020LGSVL Simulator: A High Fidelity Simulator for Autonomous Drivingcited by 119
  11. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 112
  12. 2023Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveyscited by 75
  13. 2019Risk Averse Robust Adversarial Reinforcement Learningcited by 54
  14. 2018Meta Reinforcement Learning with Latent Variable Gaussian Processescited by 44
  15. 2021Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learningcited by 41
  16. 2019SUMMIT: A Simulator for Urban Driving in Massive Mixed Trafficcited by 39
  17. 2019Curriculum Self-Paced Learning for Cross-Domain Object Detectioncited by 39
  18. 2020Formal Scenario-Based Testing of Autonomous Vehicles: From Simulation to the Real Worldcited by 38
  19. 2019Distilling Knowledge From a Deep Pose Regressor Networkcited by 34
  20. 2022PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detectioncited by 33
  21. 2022Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Trainingcited by 26
  22. 2015Adapting Deep Visuomotor Representations with Weak Pairwise Constraintscited by 25
  23. 2019Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flightcited by 25
  24. 2020Enhanced Transfer Learning for Autonomous Driving with Systematic Accident Simulationcited by 10
  25. 2022Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twincited by 9
  26. 2019Multi-Vehicle Mixed-Reality Reinforcement Learning for Autonomous Multi-Lane Drivingcited by 6
  27. 2020Sensor Fusion of Camera and Cloud Digital Twin Information for Intelligent Vehiclescited by 5
  28. 2020Augmented Reality-Based Advanced Driver-Assistance System for Connected Vehiclescited by 4
  29. 2017Transferring Autonomous Driving Knowledge on Simulated and Real Intersectionscited by 2
  30. 2022Effects of Augmented-Reality-Based Assisting Interfaces on Drivers' Object-wise Situational Awareness in Highly Autonomous Vehiclescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 20; 2025: 16; 2026: 18.

55 papers.

Sort:

By area:

  1. 2025Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Trainingcited by 39
  2. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  3. 2026DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generationcited by 36
  4. 2024WcDT: World-centric Diffusion Transformer for Traffic Scene Generationcited by 31
  5. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  6. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  7. 2024REvolve: Reward Evolution with Large Language Models using Human Feedbackcited by 22
  8. 2024Causal Composition Diffusion Model for Closed-loop Traffic Generationcited by 22
  9. 2024Are NeRFs ready for autonomous driving? Towards closing the real-to-simulation gapcited by 20
  10. 2026DriveVA: Video Action Models are Zero-Shot Driverscited by 19
  11. 2024Efficient Domain Augmentation for Autonomous Driving Testing Using Diffusion Modelscited by 10
  12. 2024Exploring Generative AI for Sim2Real in Driving Data Synthesiscited by 7
38 more
  1. 2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutionscited by 7
  2. 2025AI Simulation by Digital Twins: Systematic Survey, Reference Framework, and Mapping to a Standardized Architecturecited by 6
  3. 2024Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Drivingcited by 4
  4. 2024TinyLidarNet: 2D LiDAR-based End-to-End Deep Learning Model for F1TENTH Autonomous Racingcited by 4
  5. 2024MetaSSC: Enhancing 3D Semantic Scene Completion for Autonomous Driving through Meta-Learning and Long-sequence Modelingcited by 4
  6. 2025On the Need for a Statistical Foundation in Scenario-Based Testing of Autonomous Vehiclescited by 4
  7. 2025Industrial-Grade Sensor Simulation via Gaussian Splatting: A Modular Framework for Scalable Editing and Full-Stack Validationcited by 3
  8. 2025SimWorld: A Unified Benchmark for Simulator-Conditioned Scene Generation via World Modelcited by 3
  9. 2026DrivIng: A Large-Scale Multimodal Driving Dataset with Full Digital Twin Integrationcited by 3
  10. 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challengescited by 3
  11. 2024Learning Based NMPC Adaptation for Autonomous Driving using Parallelized Digital Twincited by 2
  12. 2024Addressing Data Annotation Challenges in Multiple Sensors: A Solution for Scania Collected Datasetscited by 2
  13. 2024Edge Case Detection in Automated Driving: Methods, Challenges, and Future Directionscited by 2
  14. 2025RALAD: Bridging the Real-to-Sim Domain Gap in Autonomous Driving with Retrieval-Augmented Learningcited by 2
  15. 2025RealMirror: A Comprehensive, Open-Source Vision-Language-Action Platform for Embodied AIcited by 2
  16. 2025LiDAR-GS++:Improving LiDAR Gaussian Reconstruction via Diffusion Priorscited by 2
  17. 2026Flexible Empowerment at Reasoning with Extended Best-of-N Samplingcited by 2
  18. 2024Behavioural gap assessment of human-vehicle interaction in real and virtual reality-based scenarios in autonomous drivingcited by 1
  19. 2024Augmented Intelligence in Smart Intersections: Local Digital Twins-Assisted Hybrid Autonomous Drivingcited by 1
  20. 2025Cutting-edge 3D reconstruction solutions for underwater coral reef images: A review and comparisoncited by 1
  21. 2025Navigating the Edge with the State-of-the-Art Insights into Corner Case Identification and Generation for Enhanced Autonomous Vehicle Safetycited by 1
  22. 2025A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systemscited by 1
  23. 2025HABIT: Human Action Benchmark for Interactive Traffic in CARLAcited by 1
  24. 2026LiDAR-EVS: Enhance Extrapolated View Synthesis for 3D Gaussian Splatting with Pseudo-LiDAR Supervisioncited by 1
  25. 2026MVAdapt: Zero-Shot Multi-Vehicle Adaptation for End-to-End Autonomous Drivingcited by 1
  26. 2024Planning the path with Reinforcement Learning: Optimal Robot Motion Planning in RoboCup Small Size League Environmentscited by 0
  27. 2024OpenConvoy: Universal Platform for Real-World Testing of Cooperative Driving Systemscited by 0
  28. 2024DiffPoGAN: Diffusion Policies with Generative Adversarial Networks for Offline Reinforcement Learningcited by 0
  29. 2024A Sim-to-Real Vision-based Lane Keeping System for a 1:10-scale Autonomous Vehiclecited by 0
  30. 2025RAFT -- A Domain Adaptation Framework for RGB & LiDAR Semantic Segmentationcited by 0
  31. 2025AI-driven visual monitoring of industrial assembly taskscited by 0
  32. 2025Real-Time Model Checking for Closed-Loop Robot Reactive Planningcited by 0
  33. 2026Motion Capture is Not the Target Domain: Scaling Synthetic Data for Learning Motion Representationscited by 0
  34. 2026Advancing Digital Twin Generation Through a Novel Simulation Framework and Quantitative Benchmarkingcited by 0
  35. 2026Developing AI Agents with Simulated Data: Why, what, and how?cited by 0
  36. 2026Learning to Tune Pure Pursuit in Autonomous Racing: Joint Lookahead and Steering-Gain Control with PPOcited by 0
  37. 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Drivingcited by 0
  38. 2026A Model-Driven Approach for Developing Families of Reinforcement Learning Environmentscited by 0

50 most cited of 55 All 55 (JSON)