# Controlling an Autonomous Vehicle with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1909.12153 · HTML: https://mlsanity.com/paper/1909.12153/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2019.

## Built on by (most cited first, 13 of 13)

- [A Survey of Deep RL and IL for Autonomous Driving Policy Learning](https://mlsanity.com/paper/2101.01993/) (2021)
- [Federated Learning for Connected and Automated Vehicles: A Survey of Existing Approaches and Challenges](https://mlsanity.com/paper/2308.10407/) (2023)
- [A Champion-level Vision-based Reinforcement Learning Agent for Competitive Racing in Gran Turismo 7](https://mlsanity.com/paper/2504.09021/) (2025)
- [Vessel-following model for inland waterways based on deep reinforcement learning](https://mlsanity.com/paper/2207.03257/) (2022)
- [Let Hybrid A* Path Planner Obey Traffic Rules: A Deep Reinforcement Learning-Based Planning Framework](https://mlsanity.com/paper/2407.01216/) (2024)
- [A Differentiated Reward Method for Reinforcement Learning based Multi-Vehicle Cooperative Decision-Making Algorithms](https://mlsanity.com/paper/2502.00352/) (2025)
- [Dynamics-Decoupled Trajectory Alignment for Sim-to-Real Transfer in Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2511.07155/) (2025)
- [Action Shapley: A Training Data Selection Metric for World Model in Reinforcement Learning](https://mlsanity.com/paper/2601.10905/) (2026)
- [An Embarrassingly Pragmatic Introduction to Vision-based Autonomous Robots](https://mlsanity.com/paper/2112.05534/) (2021)
- [Multisymplectic Formulation of Deep Learning Using Mean--Field Type Control and Nonlinear Stability of Training Algorithm](https://mlsanity.com/paper/2207.12242/) (2022)
- [Exposing the Copycat Problem of Imitation-based Planner: A Novel Closed-Loop Simulator, Causal Benchmark and Joint IL-RL Baseline](https://mlsanity.com/paper/2504.14709/) (2025)
- [CycleRL: Sim-to-Real Deep Reinforcement Learning for Robust Autonomous Bicycle Control](https://mlsanity.com/paper/2603.15013/) (2026)
- [MILER: Semantic Mid-Level Representation for Sim-to-Real Reinforcement Learning in Unstructured Autonomous Driving](https://mlsanity.com/paper/2609.20747/) (2026)

## Abstract (arXiv)

We present a control approach for autonomous vehicles based on deep reinforcement learning. A neural network agent is trained to map its estimated state to acceleration and steering commands given the objective of reaching a specific target state while considering detected obstacles. Learning is performed using state-of-the-art proximal policy optimization in combination with a simulated environment. Training from scratch takes five to nine hours. The resulting agent is evaluated within simulation and subsequently applied to control a full-size research vehicle. For this, the autonomous exploration of a parking lot is considered, including turning maneuvers and obstacle avoidance. Altogether, this work is among the first examples to successfully apply deep reinforcement learning to a real vehicle.