# Simulation-based reinforcement learning for real-world autonomous driving

arXiv: https://arxiv.org/abs/1911.12905 · HTML: https://mlsanity.com/paper/1911.12905/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2019.

## Built on by (most cited first, 30 of 52)

- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research](https://mlsanity.com/paper/2310.08710/) (2023)
- [Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients](https://mlsanity.com/paper/2109.13333/) (2021)
- [Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey](https://mlsanity.com/paper/2307.04370/) (2023)
- [WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning](https://mlsanity.com/paper/2509.13305/) (2025)
- [GINA-3D: Learning to Generate Implicit Neural Assets in the Wild](https://mlsanity.com/paper/2304.02163/) (2023)
- [Cross-Domain Policy Adaptation by Capturing Representation Mismatch](https://mlsanity.com/paper/2405.15369/) (2024)
- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-based Autonomous Urban Driving](https://mlsanity.com/paper/2202.08557/) (2022)
- [An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility](https://mlsanity.com/paper/2203.07676/) (2022)
- [Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settings](https://mlsanity.com/paper/2105.06029/) (2021)
- [Replay across Experiments: A Natural Extension of Off-Policy RL](https://mlsanity.com/paper/2311.15951/) (2023)
- [Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Data](https://mlsanity.com/paper/2210.14721/) (2022)
- [Designing Long-term Group Fair Policies in Dynamical Systems](https://mlsanity.com/paper/2311.12447/) (2023)
- [BridgeSim: Unveiling the OL-CL Gap in End-to-End Autonomous Driving](https://mlsanity.com/paper/2604.10856/) (2026)
- [Generating and Evolving Reward Functions for Highway Driving with Large Language Models](https://mlsanity.com/paper/2406.10540/) (2024)
- [LiDAR-as-Camera for End-to-End Driving](https://mlsanity.com/paper/2206.15170/) (2022)
- [Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions](https://mlsanity.com/paper/2409.13144/) (2024)
- [Regression for the Mean: Auto-Evaluation and Inference with Few Labels through Post-hoc Regression](https://mlsanity.com/paper/2411.12665/) (2024)
- [Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving](https://mlsanity.com/paper/2306.03220/) (2023)
- [ContainerGym: A Real-World Reinforcement Learning Benchmark for Resource Allocation](https://mlsanity.com/paper/2307.02991/) (2023)
- [TorchDriveEnv: A Reinforcement Learning Benchmark for Autonomous Driving with Reactive, Realistic, and Diverse Non-Playable Characters](https://mlsanity.com/paper/2405.04491/) (2024)
- [A Versatile and Efficient Reinforcement Learning Framework for Autonomous Driving](https://mlsanity.com/paper/2110.11573/) (2021)
- [Hierarchical End-to-End Autonomous Driving: Integrating BEV Perception with Deep Reinforcement Learning](https://mlsanity.com/paper/2409.17659/) (2024)
- [LEyes: A Lightweight Framework for Deep Learning-Based Eye Tracking using Synthetic Eye Images](https://mlsanity.com/paper/2309.06129/) (2023)
- [SafePath: Conformal Prediction for Safe LLM-Based Autonomous Navigation](https://mlsanity.com/paper/2505.09427/) (2025)
- [Scaling Self-Play for End-to-End Driving](https://mlsanity.com/paper/2606.19641/) (2026)
- [ARC: Adversarially Robust Control Policies for Autonomous Vehicles](https://mlsanity.com/paper/2107.04487/) (2021)
- [Soft Actor-Critic Algorithm with Truly-satisfied Inequality Constraint](https://mlsanity.com/paper/2303.04356/) (2023)

## Abstract (arXiv)

We use reinforcement learning in simulation to obtain a driving system controlling a full-size real-world vehicle. The driving policy takes RGB images from a single camera and their semantic segmentation as input. We use mostly synthetic data, with labelled real-world data appearing only in the training of the segmentation network. Using reinforcement learning in simulation and synthetic data is motivated by lowering costs and engineering effort. In real-world experiments we confirm that we achieved successful sim-to-real policy transfer. Based on the extensive evaluation, we analyze how design decisions about perception, control, and training impact the real-world performance.