mlsanityData to 14 Oct 2026

Learning to Drive from Simulation without Real World Labels

arXiv 1812.03823 ↗First posted 10 Dec 2018cs.CVcited by 39

Not on the map. Why?

Abstract (arXiv)

Simulation can be a powerful tool for understanding machine learning systems and designing methods to solve real-world problems. Training and evaluating methods purely in simulation is often "doomed to succeed" at the desired task in a simulated environment, but the resulting models are incapable of operation in the real world. Here we present and evaluate a method for transferring a vision-based lane following driving policy from simulation to operation on a rural road without any real-world labels. Our approach leverages recent advances in image-to-image translation to achieve domain transfer while jointly learning a single-camera control policy from simulation control labels. We assess the driving performance of this method using both open-loop regression metrics, and closed-loop performance operating an autonomous vehicle on rural and urban roads.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 12; 2022: 6; 2023: 9; 2024: 6; 2025: 4; 2026: 1.

39 papers.

Sort:

By area:

  1. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 493
  2. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  3. 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 137
  4. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 103
  5. 2019Urban Driving with Conditional Imitation Learningcited by 86
  6. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learningcited by 61
  7. 2022When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learningcited by 60
  8. 2021Exploring Imitation Learning for Autonomous Driving with Feedback Synthesizer and Differentiable Rasterizationcited by 22
  9. 2022Sim-to-Real Domain Adaptation for Lane Detection and Classification in Autonomous Drivingcited by 20
  10. 2021Learning Interactive Driving Policies via Data-driven Simulationcited by 17
  11. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  12. 2021Closing the Planning-Learning Loop with Application to Autonomous Drivingcited by 12
27 more
  1. 2021Towards Optimal Strategies for Training Self-Driving Perception Models in Simulationcited by 12
  2. 2021FedRAD: Federated Robust Adaptive Distillationcited by 11
  3. 2023Two is Better Than One: Digital Siblings to Improve Autonomous Driving Testingcited by 9
  4. 2024xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editingcited by 9
  5. 2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutionscited by 7
  6. 2021End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learningcited by 6
  7. 2021A Versatile and Efficient Reinforcement Learning Framework for Autonomous Drivingcited by 5
  8. 2025Sim2Real Diffusion: Leveraging Foundation Vision Language Models for Adaptive Automated Drivingcited by 5
  9. 2024Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challengescited by 3
  10. 2025No More Blind Spots: Learning Vision-Based Omnidirectional Bipedal Locomotion for Challenging Terraincited by 3
  11. 2022On the Choice of Data for Efficient Training and Validation of End-to-End Driving Modelscited by 2
  12. 2022Imitation Learning for Generalizable Self-driving Policy with Sim-to-real Transfercited by 2
  13. 2023Controlling Steering with Energy-Based Modelscited by 2
  14. 2023Parallel bootstrap-based on-policy deep reinforcement learning for continuous flow control applicationscited by 2
  15. 2023Combating the effects of speed and delays in end-to-end self-drivingcited by 2
  16. 2024End-to-End Steering for Autonomous Vehicles via Conditional Imitation Co-Learningcited by 2
  17. 2023Perception Imitation: Towards Synthesis-free Simulator for Autonomous Vehiclescited by 1
  18. 2023Attacking Motion Planners Using Adversarial Perception Errorscited by 1
  19. 2024Guiding Attention in End-to-End Driving Modelscited by 1
  20. 2021Generating Annotated Training Data for 6D Object Pose Estimation in Operational Environments with Minimal User Interactioncited by 0
  21. 2021Cycle-Consistent World Models for Domain Independent Latent Imaginationcited by 0
  22. 2022Autoencoder for Synthetic to Real Generalization: From Simple to More Complex Scenescited by 0
  23. 2022Exiting the Simulation: The Road to Robust and Resilient Autonomous Vehicles at Scalecited by 0
  24. 2023Bird's Eye View Based Pretrained World model for Visual Navigationcited by 0
  25. 2025Scalable heliostat surface predictions from focal spots: Sim-to-Real transfer of inverse Deep Learning Raytracingcited by 0
  26. 2025State-Conditional Adversarial Learning: An Off-Policy Visual Domain Transfer Method for End-to-End Imitation Learningcited by 0
  27. 2026VISTA-DZ: Visual Semantic Trajectory Adaptation for Personalized Dilemma Zone Predictioncited by 0