# Improved Reinforcement Learning through Imitation Learning Pretraining Towards Image-based Autonomous Driving

arXiv: https://arxiv.org/abs/1907.06838 · HTML: https://mlsanity.com/paper/1907.06838/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 13 of its 15 references that are in the map.
- First posted on arXiv 16 Jul 2019.

## Placed in

- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [BAM: Bottleneck Attention Module](https://mlsanity.com/paper/1807.06514/) (2018)
- [Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning](https://mlsanity.com/paper/1706.09829/) (2017)
- [Learning Accurate, Comfortable and Human-like Driving](https://mlsanity.com/paper/1903.10995/) (2019)
- [Learning Transferable UAV for Forest Visual Perception](https://mlsanity.com/paper/1806.03626/) (2018)
- [Drone Path-Following in GPS-Denied Environments using Convolutional Networks](https://mlsanity.com/paper/1905.01658/) (2019)

## Built on by (most cited first, 5 of 5)

- [DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Driving](https://mlsanity.com/paper/2403.16996/) (2024)
- [QADQN: Quantum Attention Deep Q-Network for Financial Market Prediction](https://mlsanity.com/paper/2408.03088/) (2024)
- [Emoti-Attack: Zero-Perturbation Adversarial Attacks on NLP Systems via Emoji Sequences](https://mlsanity.com/paper/2502.17392/) (2025)
- [Integrating Imitation Learning with Human Driving Data into Reinforcement Learning to Improve Training Efficiency for Autonomous Driving](https://mlsanity.com/paper/2111.11673/) (2021)
- [Learning-based Autonomous Oversteer Control and Collision Avoidance](https://mlsanity.com/paper/2505.15275/) (2025)

## Abstract (arXiv)

We present a training pipeline for the autonomous driving task given the current camera image and vehicle speed as the input to produce the throttle, brake, and steering control output. The simulator Airsim's convenient weather and lighting API provides a sufficient diversity during training which can be very helpful to increase the trained policy's robustness. In order to not limit the possible policy's performance, we use a continuous and deterministic control policy setting. We utilize ResNet-34 as our actor and critic networks with some slight changes in the fully connected layers. Considering human's mastery of this task and the high-complexity nature of this task, we first use imitation learning to mimic the given human policy and leverage the trained policy and its weights to the reinforcement learning phase for which we use DDPG. This combination shows a considerable performance boost comparing to both pure imitation learning and pure DDPG for the autonomous driving task.