# Decision-Making under On-Ramp merge Scenarios by Distributional Soft Actor-Critic Algorithm

arXiv: https://arxiv.org/abs/2103.04535 · HTML: https://mlsanity.com/paper/2103.04535/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 19 references that are in the map.
- First posted on arXiv 8 Mar 2021.

## Builds on (most cited first, 7 of 7)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge](https://mlsanity.com/paper/1709.02066/) (2017)
- [Direct and indirect reinforcement learning](https://mlsanity.com/paper/1912.10600/) (2019)
- [Hierarchical Reinforcement Learning for Self-Driving Decision-Making without Reliance on Labeled Driving Data](https://mlsanity.com/paper/2001.09816/) (2020)

## Abstract (arXiv)

Merging into the highway from the on-ramp is an essential scenario for automated driving. The decision-making under the scenario needs to balance the safety and efficiency performance to optimize a long-term objective, which is challenging due to the dynamic, stochastic, and adversarial characteristics. The Rule-based methods often lead to conservative driving on this task while the learning-based methods have difficulties meeting the safety requirements. In this paper, we propose an RL-based end-to-end decision-making method under a framework of offline training and online correction, called the Shielded Distributional Soft Actor-critic (SDSAC). The SDSAC adopts the policy evaluation with safety consideration and a safety shield parameterized with the barrier function in its offline training and online correction, respectively. These two measures support each other for better safety while not damaging the efficiency performance severely. We verify the SDSAC on an on-ramp merge scenario in simulation. The results show that the SDSAC has the best safety performance compared to baseline algorithms and achieves efficient driving simultaneously.