# Weakly Supervised Reinforcement Learning for Autonomous Highway Driving via Virtual Safety Cages

arXiv: https://arxiv.org/abs/2103.09726 · HTML: https://mlsanity.com/paper/2103.09726/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 27 of its 67 references that are in the map.
- First posted on arXiv 17 Mar 2021.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 27 of 27)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Exploring the Limitations of Behavior Cloning for Autonomous Driving](https://mlsanity.com/paper/1904.08980/) (2019)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [End-to-end Learning of Driving Models from Large-scale Video Datasets](https://mlsanity.com/paper/1612.01079/) (2016)
- [A Survey of Deep Learning Applications to Autonomous Vehicle Control](https://mlsanity.com/paper/1912.10773/) (2019)
- [Query-Efficient Imitation Learning for End-to-End Autonomous Driving](https://mlsanity.com/paper/1605.06450/) (2016)
- [A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems](https://mlsanity.com/paper/2005.02979/) (2020)
- [On Offline Evaluation of Vision-based Driving Models](https://mlsanity.com/paper/1809.04843/) (2018)
- [On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products](https://mlsanity.com/paper/1610.01256/) (2016)
- [Deep Object-Centric Policies for Autonomous Driving](https://mlsanity.com/paper/1811.05432/) (2018)
- [Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industry](https://mlsanity.com/paper/1812.05389/) (2018)
- [End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners](https://mlsanity.com/paper/1803.10158/) (2018)
- [Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints](https://mlsanity.com/paper/1904.00231/) (2019)
- [Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies](https://mlsanity.com/paper/2002.12078/) (2020)
- [Deep Reinforcement Learning with Enhanced Safety for Autonomous Highway Driving](https://mlsanity.com/paper/1910.12905/) (2019)
- [Autonomous Braking System via Deep Reinforcement Learning](https://mlsanity.com/paper/1702.02302/) (2017)
- [Weakly-Supervised Reinforcement Learning for Controllable Behavior](https://mlsanity.com/paper/2004.02860/) (2020)
- [Zero-shot Deep Reinforcement Learning Driving Policy Transfer for Autonomous Vehicles based on Robust Control](https://mlsanity.com/paper/1812.03216/) (2018)
- [Integrating Deep Reinforcement Learning with Model-based Path Planners for Automated Driving](https://mlsanity.com/paper/2002.00434/) (2020)
- [Lane-Change Initiation and Planning Approach for Highly Automated Driving on Freeways](https://mlsanity.com/paper/2007.14032/) (2020)

## Built on by (most cited first, 1 of 1)

- [Bench4Merge: A Comprehensive Benchmark for Merging in Realistic Dense Traffic with Micro-Interactive Vehicles](https://mlsanity.com/paper/2410.15912/) (2024)

## Abstract (arXiv)

The use of neural networks and reinforcement learning has become increasingly popular in autonomous vehicle control. However, the opaqueness of the resulting control policies presents a significant barrier to deploying neural network-based control in autonomous vehicles. In this paper, we present a reinforcement learning based approach to autonomous vehicle longitudinal control, where the rule-based safety cages provide enhanced safety for the vehicle as well as weak supervision to the reinforcement learning agent. By guiding the agent to meaningful states and actions, this weak supervision improves the convergence during training and enhances the safety of the final trained policy. This rule-based supervisory controller has the further advantage of being fully interpretable, thereby enabling traditional validation and verification approaches to ensure the safety of the vehicle. We compare models with and without safety cages, as well as models with optimal and constrained model parameters, and show that the weak supervision consistently improves the safety of exploration, speed of convergence, and model performance. Additionally, we show that when the model parameters are constrained or sub-optimal, the safety cages can enable a model to learn a safe driving policy even when the model could not be trained to drive through reinforcement learning alone.