# Safe Reinforcement Learning with Scene Decomposition for Navigating Complex Urban Environments

arXiv: https://arxiv.org/abs/1904.11483 · HTML: https://mlsanity.com/paper/1904.11483/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2019.

## Built on by (most cited first, 18 of 18)

- [A Survey of Deep RL and IL for Autonomous Driving Policy Learning](https://mlsanity.com/paper/2101.01993/) (2021)
- [Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning](https://mlsanity.com/paper/2107.07316/) (2021)
- [Occlusion-Aware Crowd Navigation Using People as Sensors](https://mlsanity.com/paper/2210.00552/) (2022)
- [Multi-task Safe Reinforcement Learning for Navigating Intersections in Dense Traffic](https://mlsanity.com/paper/2202.09644/) (2022)
- [Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions](https://mlsanity.com/paper/2409.13144/) (2024)
- [Safe Deep Q-Network for Autonomous Vehicles at Unsignalized Intersection](https://mlsanity.com/paper/2106.04561/) (2021)
- [Multi-Task Conditional Imitation Learning for Autonomous Navigation at Crowded Intersections](https://mlsanity.com/paper/2202.10124/) (2022)
- [Decision Making in Non-Stationary Environments with Policy-Augmented Search](https://mlsanity.com/paper/2401.03197/) (2024)
- [Autonomous Navigation through intersections with Graph ConvolutionalNetworks and Conditional Imitation Learning for Self-driving Cars](https://mlsanity.com/paper/2102.00675/) (2021)
- [Generalizing Decision Making for Automated Driving with an Invariant Environment Representation using Deep Reinforcement Learning](https://mlsanity.com/paper/2102.06765/) (2021)
- [Pedestrian Collision Avoidance for Autonomous Vehicles at Unsignalized Intersection Using Deep Q-Network](https://mlsanity.com/paper/2105.00153/) (2021)
- [A Safety-Critical Decision Making and Control Framework Combining Machine Learning and Rule-based Algorithms](https://mlsanity.com/paper/2201.12819/) (2022)
- [Integrating occlusion awareness in urban motion prediction for enhanced autonomous vehicle navigation](https://mlsanity.com/paper/2406.19798/) (2024)
- [Automatically Learning Fallback Strategies with Model-Free Reinforcement Learning in Safety-Critical Driving Scenarios](https://mlsanity.com/paper/2204.05196/) (2022)
- [Safer Autonomous Driving in a Stochastic, Partially-Observable Environment by Hierarchical Contingency Planning](https://mlsanity.com/paper/2204.06509/) (2022)
- [Belief Aided Navigation using Bayesian Reinforcement Learning for Avoiding Humans in Blind Spots](https://mlsanity.com/paper/2403.10105/) (2024)
- [Enhancing Safety for Autonomous Agents in Partly Concealed Urban Traffic Environments Through Representation-Based Shielding](https://mlsanity.com/paper/2407.04343/) (2024)
- [Variational Quantum Rainbow Deep Q-Network for Optimizing Resource Allocation Problem](https://mlsanity.com/paper/2512.05946/) (2025)

## Abstract (arXiv)

Navigating urban environments represents a complex task for automated vehicles. They must reach their goal safely and efficiently while considering a multitude of traffic participants. We propose a modular decision making algorithm to autonomously navigate intersections, addressing challenges of existing rule-based and reinforcement learning (RL) approaches. We first present a safe RL algorithm relying on a model-checker to ensure safety guarantees. To make the decision strategy robust to perception errors and occlusions, we introduce a belief update technique using a learning based approach. Finally, we use a scene decomposition approach to scale our algorithm to environments with multiple traffic participants. We empirically demonstrate that our algorithm outperforms rule-based methods and reinforcement learning techniques on a complex intersection scenario.