# Autonomous Driving with Deep Reinforcement Learning in CARLA Simulation

arXiv: https://arxiv.org/abs/2306.11217 · HTML: https://mlsanity.com/paper/2306.11217/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 20 Jun 2023.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Watch This: Scalable Cost-Function Learning for Path Planning in Urban Environments](https://mlsanity.com/paper/1607.02329/) (2016)
- [Driving in Real Life with Inverse Reinforcement Learning](https://mlsanity.com/paper/2206.03004/) (2022)
- [Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks](https://mlsanity.com/paper/1612.03653/) (2016)
- [Using Graph-Theoretic Machine Learning to Predict Human Driver Behavior](https://mlsanity.com/paper/2111.02964/) (2021)

## Built on by (most cited first, 5 of 5)

- [When Should We Prefer State-to-Visual DAgger Over Visual Reinforcement Learning?](https://mlsanity.com/paper/2412.13662/) (2024)
- [HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agents](https://mlsanity.com/paper/2503.08241/) (2025)
- [Generative AI for Autonomous Driving: A Review](https://mlsanity.com/paper/2505.15863/) (2025)
- [3CSim: CARLA Corner Case Simulation for Control Assessment in Autonomous Driving](https://mlsanity.com/paper/2409.10524/) (2024)
- [Lifting Vision: Ground to Aerial Localization with Reasoning Guided Planning](https://mlsanity.com/paper/2512.24404/) (2025)

## Abstract (arXiv)

Nowadays, autonomous vehicles are gaining traction due to their numerous potential applications in resolving a variety of other real-world challenges. However, developing autonomous vehicles need huge amount of training and testing before deploying it to real world. While the field of reinforcement learning (RL) has evolved into a powerful learning framework to the development of deep representation learning, and it is now capable of learning complicated policies in high-dimensional environments like in autonomous vehicles. In this regard, we make an effort, using Deep Q-Learning, to discover a method by which an autonomous car may maintain its lane at top speed while avoiding other vehicles. After that, we used CARLA simulation environment to test and verify our newly acquired policy based on the problem formulation.