mlsanityData to 7 Oct 2026

A Survey of Deep RL and IL for Autonomous Driving Policy Learning

arXiv 2101.01993 ↗First posted 6 Jan 2021cs.ROcited by 59

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Autonomous driving (AD) agents generate driving policies based on online perception results, which are obtained at multiple levels of abstraction, e.g., behavior planning, motion planning and control. Driving policies are crucial to the realization of safe, efficient and harmonious driving behaviors, where AD agents still face substantial challenges in complex scenarios. Due to their successful application in fields such as robotics and video games, the use of deep reinforcement learning (DRL) and deep imitation learning (DIL) techniques to derive AD policies have witnessed vast research efforts in recent years. This paper is a comprehensive survey of this body of work, which is conducted at three levels: First, a taxonomy of the literature studies is constructed from the system perspective, among which five modes of integration of DRL/DIL models into an AD architecture are identified. Second, the formulations of DRL/DIL models for conducting specified AD tasks are comprehensively reviewed, where various designs on the model state and action spaces and the reinforcement learning rewards are covered. Finally, an in-depth review is conducted on how the critical issues of AD applications regarding driving safety, interaction with other traffic participants and uncertainty of the environment are addressed by the DRL/DIL models. To the best of our knowledge, this is the first survey to focus on AD policy learning using DRL/DIL, which is addressed simultaneously from the system, task-driven and problem-driven perspectives. We share and discuss findings, which may lead to the investigation of various topics in the future.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 123 of its 245 references that are in the map. First posted on arXiv 6 Jan 2021.

Builds on

123 of 245 references.

References · 123

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  7. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  8. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  9. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  10. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  11. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  12. 2016Generative Adversarial Imitation Learningcited by 1,576
38 more
  1. 2017Deep Setscited by 1,394
  2. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  4. 2015Prioritized Experience Replaycited by 1,070
  5. 2016End to End Learning for Self-Driving Carscited by 1,000
  6. 2017Deep Bayesian Active Learning with Image Datacited by 812
  7. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  8. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  9. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  10. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  11. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  12. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  13. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  14. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  15. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  16. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  17. 2018A Survey on Deep Transfer Learningcited by 417
  18. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
  19. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  20. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  21. 2016Learning to reinforcement learncited by 369
  22. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  23. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  24. 2017Deep Reinforcement Learning: An Overviewcited by 327
  25. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  26. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
  27. 2018Learning to Drive in a Daycited by 266
  28. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  29. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
  30. 2018Reinforced Continual Learningcited by 186
  31. 2017Social Attention: Modeling Attention in Human Crowdscited by 185
  32. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  33. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  34. 2018Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learningcited by 162
  35. 2015Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inferencecited by 159
  36. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 149
  37. 2018Graph Convolutional Reinforcement Learningcited by 148
  38. 2016End-to-end Learning of Driving Models from Large-scale Video Datasetscited by 147

50 most cited of 123 All 123 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 8; 2023: 17; 2024: 15; 2025: 10; 2026: 5.

59 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 112
  3. 2021Reward (Mis)design for Autonomous Drivingcited by 102
  4. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  5. 2023CAT: Closed-loop Adversarial Training for Safe End-to-End Drivingcited by 52
  6. 2024HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Drivingcited by 31
  7. 2021Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Drivingcited by 17
  8. 2024Research on Autonomous Driving Decision-making Strategies based Deep Reinforcement Learningcited by 17
  9. 2021DQ-GAT: Towards Safe and Efficient Autonomous Driving with Deep Q-Learning and Graph Attention Networkscited by 13
  10. 2023Dynamically Conservative Self-Driving Planner for Long-Tail Casescited by 13
  11. 2025TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learningcited by 13
  12. 2022Cooperative Behavior Planning for Automated Driving using Graph Neural Networkscited by 8
38 more
  1. 2023Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrationscited by 8
  2. 2023Goal-Guided Transformer-Enabled Reinforcement Learning for Efficient Autonomous Navigationcited by 7
  3. 2023Action and Trajectory Planning for Urban Autonomous Driving with Hierarchical Reinforcement Learningcited by 7
  4. 2023End-to-end Autonomous Driving using Deep Learning: A Systematic Reviewcited by 7
  5. 2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutionscited by 7
  6. 2025Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Drivingcited by 7
  7. 2022Risk-Sensitive Policy with Distributional Reinforcement Learningcited by 6
  8. 2023Nav-Q: Quantum Deep Reinforcement Learning for Collision-Free Navigation of Self-Driving Carscited by 6
  9. 2024Knowledge Transfer from Simple to Complex: A Safe and Efficient Reinforcement Learning Framework for Autonomous Driving Decision-Makingcited by 5
  10. 2022Multi-Task Conditional Imitation Learning for Autonomous Navigation at Crowded Intersectionscited by 4
  11. 2023Integration of Reinforcement Learning Based Behavior Planning With Sampling Based Motion Planning for Automated Drivingcited by 4
  12. 2024Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Drivingcited by 4
  13. 2026FlowAD: Ego-Scene Interactive Modeling for Autonomous Drivingcited by 4
  14. 2021Modeling Interactions of Autonomous Vehicles and Pedestrians with Deep Multi-Agent Reinforcement Learning for Collision Avoidancecited by 3
  15. 2022An Enhanced Graph Representation for Machine Learning Based Automatic Intersection Managementcited by 3
  16. 2022Hybrid Deep Reinforcement Learning and Planning for Safe and Comfortable Automated Drivingcited by 3
  17. 2023Automatic Intersection Management in Mixed Traffic Using Reinforcement Learning and Graph Neural Networkscited by 3
  18. 2023Imagination-Augmented Hierarchical Reinforcement Learning for Safe and Interactive Autonomous Driving in Urban Environmentscited by 3
  19. 2024Compressing Deep Reinforcement Learning Networks with a Dynamic Structured Pruning Method for Autonomous Drivingcited by 3
  20. 2024BECAUSE: Bilinear Causal Representation for Generalizable Offline Model-based Reinforcement Learningcited by 3
  21. 2024Importance Sampling-Guided Meta-Training for Intelligent Agents in Highly Interactive Environmentscited by 3
  22. 2024PP-TIL: Personalized Planning for Autonomous Driving with Instance-based Transfer Imitation Learningcited by 3
  23. 2022A Bibliometric Analysis and Review on Reinforcement Learning for Transportation Applicationscited by 2
  24. 2023Comprehensive Training and Evaluation on Deep Reinforcement Learning for Automated Driving in Various Simulated Driving Maneuverscited by 2
  25. 2023An Overview about Emerging Technologies of Autonomous Drivingcited by 2
  26. 2024pyrtklib: An open-source package for tightly coupled deep learning and GNSS integration for positioning in urban canyonscited by 2
  27. 2024Automated Driving with Evolution Capability: A Reinforcement Learning Method with Monotonic Performance Enhancementcited by 2
  28. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
  29. 2026Robustness Is a Function, Not a Number: A Factorized Comprehensive Study of OOD Robustness in Vision-Based Drivingcited by 2
  30. 2022GFCL: A GRU-based Federated Continual Learning Framework against Data Poisoning Attacks in IoVcited by 1
  31. 2022Rationale-aware Autonomous Driving Policy utilizing Safety Force Field implemented on CARLA Simulatorcited by 1
  32. 2024Mix Q-learning for Lane Changing: A Collaborative Decision-Making Method in Multi-Agent Deep Reinforcement Learningcited by 1
  33. 2024Towards Safe and Robust Autonomous Vehicle Platooning: A Self-Organizing Cooperative Control Frameworkcited by 1
  34. 2025ME3-BEV: Mamba-Enhanced Deep Reinforcement Learning for End-to-End Autonomous Driving with BEV-Perceptioncited by 1
  35. 2026Markov Potential Game and Multi-Agent Reinforcement Learning for Autonomous Drivingcited by 1
  36. 2023Predictive Maneuver Planning with Deep Reinforcement Learning (PMP-DRL) for comfortable and safe autonomous drivingcited by 0
  37. 2023Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehiclecited by 0
  38. 2024Safe Policy Exploration Improvement via Subgoalscited by 0

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Review of Deep Reinforcement Learning for Autonomous Driving
  2. 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving
  3. 2023Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approach
  4. 2021A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles
  5. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator
  6. 2023Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer
  7. 2021Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving
  8. 2025Evaluating Scenario-based Decision-making for Interactive Autonomous Driving Using Rational Criteria: A Survey