# Towards Robust On-Ramp Merging via Augmented Multimodal Reinforcement Learning

arXiv: https://arxiv.org/abs/2208.07307 · HTML: https://mlsanity.com/paper/2208.07307/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 49 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [The Effectiveness of Data Augmentation in Image Classification using Deep Learning](https://mlsanity.com/paper/1712.04621/) (2017)
- [Multimodal End-to-End Autonomous Driving](https://mlsanity.com/paper/1906.03199/) (2019)
- [A Survey of Deep Reinforcement Learning in Video Games](https://mlsanity.com/paper/1912.10944/) (2019)
- [Learning to Perform Physics Experiments via Deep Reinforcement Learning](https://mlsanity.com/paper/1611.01843/) (2016)
- [Pay "Attention" to Adverse Weather: Weather-aware Attention-based Object Detection](https://mlsanity.com/paper/2204.10803/) (2022)
- [A Deep Reinforcement Learning Architecture for Multi-stage Optimal Control](https://mlsanity.com/paper/1911.10684/) (2019)

## Abstract (arXiv)

Despite the success of AI-enabled onboard perception, on-ramp merging has been one of the main challenges for autonomous driving. Due to limited sensing range of onboard sensors, a merging vehicle can hardly observe main road conditions and merge properly. By leveraging the wireless communications between connected and automated vehicles (CAVs), a merging CAV has potential to proactively obtain the intentions of nearby vehicles. However, CAVs can be prone to inaccurate observations, such as the noisy basic safety messages (BSM) and poor quality surveillance images. In this paper, we present a novel approach for Robust on-ramp merge of CAVs via Augmented and Multi-modal Reinforcement Learning, named by RAMRL. Specifically, we formulate the on-ramp merging problem as a Markov decision process (MDP) by taking driving safety, comfort driving behavior, and traffic efficiency into account. To provide reliable merging maneuvers, we simultaneously leverage BSM and surveillance images for multi-modal observation, which is used to learn a policy model through proximal policy optimization (PPO). Moreover, to improve data efficiency and provide better generalization performance, we train the policy model with augmented data (e.g., noisy BSM and noisy surveillance images). Extensive experiments are conducted with Simulation of Urban MObility (SUMO) platform under two typical merging scenarios. Experimental results demonstrate the effectiveness and efficiency of our robust on-ramp merging design.