# RACE-SM: Reinforcement Learning Based Autonomous Control for Social On-Ramp Merging

arXiv: https://arxiv.org/abs/2403.03359 · HTML: https://mlsanity.com/paper/2403.03359/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 36 references that are in the map.
- First posted on arXiv 5 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Interaction-aware Decision-making for Automated Vehicles using Social Value Orientation](https://mlsanity.com/paper/2207.05853/) (2022)
- [Cooperative Autonomous Vehicles that Sympathize with Human Drivers](https://mlsanity.com/paper/2107.00898/) (2021)
- [Interaction-Aware Trajectory Prediction and Planning for Autonomous Vehicles in Forced Merge Scenarios](https://mlsanity.com/paper/2112.07624/) (2021)
- [Autonomous Highway Merging in Mixed Traffic Using Reinforcement Learning and Motion Predictive Safety Controller](https://mlsanity.com/paper/2204.01024/) (2022)
- [Merging Control Strategies of Connected and Autonomous Vehicles at Freeway On-Ramps: A Comprehensive Review](https://mlsanity.com/paper/2202.09457/) (2022)
- [Combining Reinforcement Learning with Model Predictive Control for On-Ramp Merging](https://mlsanity.com/paper/2011.08484/) (2020)

## Abstract (arXiv)

Autonomous parallel-style on-ramp merging in human controlled traffic continues to be an existing issue for autonomous vehicle control. Existing non-learning based solutions for vehicle control rely on rules and optimization primarily. These methods have been seen to present significant challenges. Recent advancements in Deep Reinforcement Learning have shown promise and have received significant academic interest however the available learning based approaches show inadequate attention to other highway vehicles and often rely on inaccurate road traffic assumptions. In addition, the parallel-style case is rarely considered. A novel learning based model for acceleration and lane change decision making that explicitly considers the utility to both the ego vehicle and its surrounding vehicles which may be cooperative or uncooperative to produce behaviour that is socially acceptable is proposed. The novel reward function makes use of Social Value Orientation to weight the vehicle's level of social cooperation and is divided into ego vehicle and surrounding vehicle utility which are weighted according to the model's designated Social Value Orientation. A two-lane highway with an on-ramp divided into a taper-style and parallel-style section is considered. Simulation results indicated the importance of considering surrounding vehicles in reward function design and show that the proposed model matches or surpasses those in literature in terms of collisions while also introducing socially courteous behaviour avoiding near misses and anti-social behaviour through direct consideration of the effect of merging on surrounding vehicles.