mlsanityData to 14 Oct 2026

Learning to Drive Using Sparse Imitation Reinforcement Learning

arXiv 2205.12128 ↗First posted 24 May 2022cs.CVcited by 1

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

In this paper, we propose Sparse Imitation Reinforcement Learning (SIRL), a hybrid end-to-end control policy that combines the sparse expert driving knowledge with reinforcement learning (RL) policy for autonomous driving (AD) task in CARLA simulation environment. The sparse expert is designed based on hand-crafted rules which is suboptimal but provides a risk-averse strategy by enforcing experience for critical scenarios such as pedestrian and vehicle avoidance, and traffic light detection. As it has been demonstrated, training a RL agent from scratch is data-inefficient and time consuming particularly for the urban driving task, due to the complexity of situations stemming from the vast size of state space. Our SIRL strategy provides a solution to solve these problems by fusing the output distribution of the sparse expert policy and the RL policy to generate a composite driving policy. With the guidance of the sparse expert during the early training stage, SIRL strategy accelerates the training process and keeps the RL exploration from causing a catastrophe outcome, and ensures safe exploration. To some extent, the SIRL agent is imitating the driving expert's behavior. At the same time, it continuously gains knowledge during training therefore it keeps making improvement beyond the sparse expert, and can surpass both the sparse expert and a traditional RL agent. We experimentally validate the efficacy of proposed SIRL approach in a complex urban scenario within the CARLA simulator. Besides, we compare the SIRL agent's performance for risk-averse exploration and high learning efficiency with the traditional RL approach. We additionally demonstrate the SIRL agent's generalization ability to transfer the driving skill to unseen environment.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 17 of its 24 references that are in the map. First posted on arXiv 24 May 2022.

Builds on

17 of 24 references.

References · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2017End-to-end Driving via Conditional Imitation Learningcited by 485
  6. 2019Learning by Cheatingcited by 383
  7. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 377
  8. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 346
  9. 2018Residual Reinforcement Learning for Robot Controlcited by 274
  10. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  11. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 194
  12. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
5 more
  1. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 149
  2. 2021Learning to drive from a world on railscited by 114
  3. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 97
  4. 2018Conditional Affordance Learning for Driving in Urban Environmentscited by 70
  5. 2021Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Roboticscited by 20

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulatorcited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2021Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving
  2. 2020Guided Policy Search Model-based Reinforcement Learning for Urban Autonomous Driving
  3. 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving
  4. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors
  5. 2024CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Driving
  6. 2020Deep Reinforcement Learning for Autonomous Driving: A Survey
  7. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learning
  8. 2020Learning to Drive (L2D) as a Low-Cost Benchmark for Real-World Reinforcement Learning