mlsanityData to 14 Oct 2026

CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Driving

arXiv 2406.08878 ↗First posted 13 Jun 2024cs.LGcited by 12

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

also in 1 more
Abstract (arXiv)

Modern approaches to autonomous driving rely heavily on learned components trained with large amounts of human driving data via imitation learning. However, these methods require large amounts of expensive data collection and even then face challenges with safely handling long-tail scenarios and compounding errors over time. At the same time, pure Reinforcement Learning (RL) methods can fail to learn performant policies in sparse, constrained, and challenging-to-define reward settings such as autonomous driving. Both of these challenges make deploying purely cloned or pure RL policies in safety critical applications such as autonomous vehicles challenging. In this paper we propose Combining IMitation and Reinforcement Learning (CIMRL) approach - a safe reinforcement learning framework that enables training driving policies in simulation through leveraging imitative motion priors and safety constraints. CIMRL does not require extensive reward specification and improves on the closed loop behavior of pure cloning methods. By combining RL and imitation, we demonstrate that our method achieves state-of-the-art results in closed loop simulation and real world driving benchmarks.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 29 of its 46 references that are in the map. First posted on arXiv 13 Jun 2024.

Builds on

29 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research12 Oct
  2. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios21 Dec

Other references · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  3. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  5. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  6. 2016End to End Learning for Self-Driving Carscited by 1,009
  7. 2017Constrained Policy Optimizationcited by 983
  8. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  9. 2017End-to-end Driving via Conditional Imitation Learningcited by 485
  10. 2017Safe Reinforcement Learning via Shieldingcited by 449
  11. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  12. 2018Reward Constrained Policy Optimizationcited by 417
15 more
  1. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  2. 2020TNT: Target-driveN Trajectory Predictioncited by 347
  3. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 280
  4. 2018Learning to Drive in a Daycited by 273
  5. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 270
  6. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  7. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
  8. 2019Soft Actor-Critic for Discrete Action Settingscited by 174
  9. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 140
  10. 2015Policy Gradient for Coherent Risk Measurescited by 76
  11. 2022Efficient Risk-Averse Reinforcement Learningcited by 43
  12. 2017Multi-step Reinforcement Learning: A Unifying Algorithmcited by 42
  13. 2016Funnel Libraries for Real-Time Robust Feedback Motion Planningcited by 29
  14. 2018Bridging the Gap Between Safety and Real-Time Performance in Receding-Horizon Trajectory Design for Mobile Robotscited by 11
  15. 2017Safe Trajectory Synthesis for Autonomous Driving in Unforeseen Environmentscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 1.

12 papers.

  1. 2025DiWA: Diffusion Policy Adaptation with World Modelscited by 31
  2. 2025DexTrack: Towards Generalizable Neural Tracking Control for Dexterous Manipulation from Human Referencescited by 17
  3. 2025Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorercited by 16
  4. 2025A Unified Framework for Real-Time Failure Handling in Robotics Using Vision-Language Models, Reactive Planner and Behavior Treescited by 4
  5. 2025HAD-Gen: Human-like and Diverse Driving Behavior Modeling for Controllable Scenario Generationcited by 3
  6. 2026Beyond Self-Play: Hierarchical Reasoning for Continuous Motion in Closed-Loop Traffic Simulationcited by 1
  7. 2025Towards a Reward-Free Reinforcement Learning Framework for Vehicle Controlcited by 0
  8. 2025CAPS: Context-Aware Priority Sampling for Enhanced Imitation Learning in Autonomous Drivingcited by 0
  9. 2025Exposing the Copycat Problem of Imitation-based Planner: A Novel Closed-Loop Simulator, Causal Benchmark and Joint IL-RL Baselinecited by 0
  10. 2025Robust Planning for Autonomous Driving via Mixed Adversarial Diffusion Predictionscited by 0
  11. 2025Benchmarking Autonomous Driving Planners Across Leaderboards: A Unified CARLA-Based Evaluationcited by 0
  12. 2025Learning Safe Autonomous Driving Policies Using Predictive Safety Representationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving
  2. 2025From Imitation to Optimization: A Comparative Study of Offline Learning for Autonomous Driving
  3. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving
  4. 2022Exploring the trade off between human driving imitation and safety for traffic simulation
  5. 2022Learning from Demonstrations of Critical Driving Behaviours Using Driver's Risk Field
  6. 2020Guided Policy Search Model-based Reinforcement Learning for Urban Autonomous Driving
  7. 2020Safe Trajectory Planning Using Reinforcement Learning for Self Driving
  8. 2024IGDrivSim: A Benchmark for the Imitation Gap in Autonomous Driving