# Safe, Efficient, Comfort, and Energy-saving Automated Driving through Roundabout Based on Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2306.11465 · HTML: https://mlsanity.com/paper/2306.11465/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 7 of its 19 references that are in the map.
- First posted on arXiv 20 Jun 2023.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 7 of 7)

- [A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers](https://mlsanity.com/paper/1804.07871/) (2018)
- [Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic](https://mlsanity.com/paper/2105.05701/) (2021)
- [End-to-End Deep Reinforcement Learning for Lane Keeping Assist](https://mlsanity.com/paper/1612.04340/) (2016)
- [Risk-Aware High-level Decisions for Automated Driving at Occluded Intersections with Reinforcement Learning](https://mlsanity.com/paper/2004.04450/) (2020)
- [Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge](https://mlsanity.com/paper/1709.02066/) (2017)
- [Learning Negotiating Behavior Between Cars in Intersections using Deep Q-Learning](https://mlsanity.com/paper/1810.10469/) (2018)
- [Intelligent Roundabout Insertion using Deep Reinforcement Learning](https://mlsanity.com/paper/2001.00786/) (2020)

## Built on by (most cited first, 3 of 3)

- [Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions](https://mlsanity.com/paper/2409.13144/) (2024)
- [Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges](https://mlsanity.com/paper/2603.09086/) (2026)
- [Risk-Aware Reinforcement Learning for Autonomous Driving: Improving Safety When Driving through Intersection](https://mlsanity.com/paper/2503.19690/) (2025)

## Abstract (arXiv)

Traffic scenarios in roundabouts pose substantial complexity for automated driving. Manually mapping all possible scenarios into a state space is labor-intensive and challenging. Deep reinforcement learning (DRL) with its ability to learn from interacting with the environment emerges as a promising solution for training such automated driving models. This study explores, employs, and implements various DRL algorithms, namely Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Trust Region Policy Optimization (TRPO) to instruct automated vehicles' driving through roundabouts. The driving state space, action space, and reward function are designed. The reward function considers safety, efficiency, comfort, and energy consumption to align with real-world requirements. All three tested DRL algorithms succeed in enabling automated vehicles to drive through the roundabout. To holistically evaluate the performance of these algorithms, this study establishes an evaluation methodology considering multiple indicators such as safety, efficiency, and comfort level. A method employing the Analytic Hierarchy Process is also developed to weigh these evaluation indicators. Experimental results on various testing scenarios reveal that the TRPO algorithm outperforms DDPG and PPO in terms of safety and efficiency, and PPO performs best in terms of comfort level. Lastly, to verify the model's adaptability and robustness regarding other driving scenarios, this study also deploys the model trained by TRPO to a range of different testing scenarios, e.g., highway driving and merging. Experimental results demonstrate that the TRPO model trained on only roundabout driving scenarios exhibits a certain degree of proficiency in highway driving and merging scenarios. This study provides a foundation for the application of automated driving with DRL in real traffic environments.