# Self-driving scale car trained by Deep reinforcement learning

arXiv: https://arxiv.org/abs/1909.03467 · HTML: https://mlsanity.com/paper/1909.03467/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Sep 2019.

## Built on by (most cited first, 5 of 5)

- [Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driving Systems](https://mlsanity.com/paper/2112.11255/) (2021)
- [Testing of Deep Reinforcement Learning Agents with Surrogate Models](https://mlsanity.com/paper/2305.12751/) (2023)
- [Boundary State Generation for Testing and Improvement of Autonomous Driving Systems](https://mlsanity.com/paper/2307.10590/) (2023)
- [A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systems](https://mlsanity.com/paper/2509.22379/) (2025)
- [Mini Autonomous Car Driving based on 3D Convolutional Neural Networks](https://mlsanity.com/paper/2508.21271/) (2025)

## Abstract (arXiv)

The self-driving based on deep reinforcement learning, as the most important application of artificial intelligence, has become a popular topic. Most of the current self-driving methods focus on how to directly learn end-to-end self-driving control strategy from the raw sensory data. Essentially, this control strategy can be considered as a mapping between images and driving behavior, which usually faces a problem of low generalization ability. To improve the generalization ability for the driving behavior, the reinforcement learning method requires extrinsic reward from the real environment, which may damage the car. In order to obtain a good generalization ability in safety, a virtual simulation environment that can be constructed different driving scene is designed by Unity. A theoretical model is established and analyzed in the virtual simulation environment, and it is trained by double Deep Q-network. Then, the trained model is migrated to a scale car in real world. This process is also called a sim2real method. The sim2real training method efficiently handle the these two problems. The simulations and experiments are carried out to evaluate the performance and effectiveness of the proposed algorithm. Finally, it is demonstrated that the scale car in real world obtain the capability for autonomous driving.