# Autonomous Driving using Residual Sensor Fusion and Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2312.16620 · HTML: https://mlsanity.com/paper/2312.16620/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 27 Dec 2023.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Learning to Walk via Deep Reinforcement Learning](https://mlsanity.com/paper/1812.11103/) (2018)
- [Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving](https://mlsanity.com/paper/2210.09539/) (2022)
- [Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2011.04752/) (2020)
- [Multi-modal Fusion Technology based on Vehicle Information: A Survey](https://mlsanity.com/paper/2211.06080/) (2022)
- [AVOID: Autonomous Vehicle Operation Incident Dataset Across the Globe](https://mlsanity.com/paper/2303.12889/) (2023)
- [MPC Builder for Autonomous Drive: Automatic Generation of MPCs for Motion Planning and Control](https://mlsanity.com/paper/2210.16512/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator](https://mlsanity.com/paper/2509.08221/) (2025)
- [Multi-Head Attention-Based Feature Extractor Integration with Soft Actor-Critic for Porosity Prediction and Process Parameter Optimization in Additive Manufacturing](https://mlsanity.com/paper/2606.20087/) (2026)

## Abstract (arXiv)

This paper proposes a novel approach by integrating sensor fusion with deep reinforcement learning, specifically the Soft Actor-Critic (SAC) algorithm, to develop an optimal control policy for self-driving cars. Our system employs a two-branch fusion method for vehicle image and tracking sensor data, leveraging the strengths of residual structures and identity mapping to enhance agent training. Through comprehensive comparisons, we demonstrate the efficacy of information fusion and establish the superiority of our selected algorithm over alternative approaches. Our work advances the field of autonomous driving and demonstrates the potential of reinforcement learning in enabling intelligent vehicle decision-making.