# Virtual to Real Reinforcement Learning for Autonomous Driving

arXiv: https://arxiv.org/abs/1704.03952 · HTML: https://mlsanity.com/paper/1704.03952/

## Key facts

- Cited by 80 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2017.

## Built on by (most cited first, 30 of 80)

- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios](https://mlsanity.com/paper/2212.11419/) (2022)
- [Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients](https://mlsanity.com/paper/2109.13333/) (2021)
- [Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey](https://mlsanity.com/paper/2307.04370/) (2023)
- [Street-View Image Generation from a Bird's-Eye View Layout](https://mlsanity.com/paper/2301.04634/) (2023)
- [VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles](https://mlsanity.com/paper/2111.12083/) (2021)
- [Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving](https://mlsanity.com/paper/2210.09539/) (2022)
- [Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation](https://mlsanity.com/paper/2102.05714/) (2021)
- [Towards Knowledge-driven Autonomous Driving](https://mlsanity.com/paper/2312.04316/) (2023)
- [Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learning](https://mlsanity.com/paper/2209.08483/) (2022)
- [Learning Realistic Traffic Agents in Closed-loop](https://mlsanity.com/paper/2311.01394/) (2023)
- [CFlowNets: Continuous Control with Generative Flow Networks](https://mlsanity.com/paper/2303.02430/) (2023)
- [A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning](https://mlsanity.com/paper/2101.06778/) (2021)
- [On the Foundation of Distributionally Robust Reinforcement Learning](https://mlsanity.com/paper/2311.09018/) (2023)
- [A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving](https://mlsanity.com/paper/2405.01440/) (2024)
- [Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving](https://mlsanity.com/paper/2105.10266/) (2021)
- [Learning Interactive Driving Policies via Data-driven Simulation](https://mlsanity.com/paper/2111.12137/) (2021)
- [Human-compatible driving partners through data-regularized self-play reinforcement learning](https://mlsanity.com/paper/2403.19648/) (2024)
- [A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents](https://mlsanity.com/paper/2402.04580/) (2024)
- [SENTINEL: Taming Uncertainty with Ensemble-based Distributional Reinforcement Learning](https://mlsanity.com/paper/2102.11075/) (2021)
- [Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art](https://mlsanity.com/paper/2308.16316/) (2023)
- [Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarios](https://mlsanity.com/paper/2210.02300/) (2022)
- [Learning Robotic Navigation from Experience: Principles, Methods, and Recent Results](https://mlsanity.com/paper/2212.06759/) (2022)
- [Return Augmented Decision Transformer for Off-Dynamics Reinforcement Learning](https://mlsanity.com/paper/2410.23450/) (2024)
- [Diffusion Based Augmentation for Captioning and Retrieval in Cultural Heritage](https://mlsanity.com/paper/2308.07151/) (2023)
- [Gaze Training by Modulated Dropout Improves Imitation Learning](https://mlsanity.com/paper/1904.08377/) (2019)
- [A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles](https://mlsanity.com/paper/2105.14218/) (2021)
- [Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin](https://mlsanity.com/paper/2211.14874/) (2022)
- [Quantum Advantage Actor-Critic for Reinforcement Learning](https://mlsanity.com/paper/2401.07043/) (2024)

## Abstract (arXiv)

Reinforcement learning is considered as a promising direction for driving policy learning. However, training autonomous driving vehicle with reinforcement learning in real environment involves non-affordable trial-and-error. It is more desirable to first train in a virtual environment and then transfer to the real environment. In this paper, we propose a novel realistic translation network to make model trained in virtual environment be workable in real world. The proposed network can convert non-realistic virtual image input into a realistic one with similar scene structure. Given realistic frames as input, driving policy trained by reinforcement learning can nicely adapt to real world driving. Experiments show that our proposed virtual to real (VR) reinforcement learning (RL) works pretty well. To our knowledge, this is the first successful case of driving policy trained by reinforcement learning that can adapt to real world driving data.