mlsanityData to 7 Oct 2026

Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous Driving

arXiv 2104.07246 ↗First posted 15 Apr 2021cs.ROcited by 16

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Due to the limited smartness and abilities of machine intelligence, currently autonomous vehicles are still unable to handle all kinds of situations and completely replace drivers. Because humans exhibit strong robustness and adaptability in complex driving scenarios, it is of great importance to introduce humans into the training loop of artificial intelligence, leveraging human intelligence to further advance machine learning algorithms. In this study, a real-time human-guidance-based deep reinforcement learning (Hug-DRL) method is developed for policy training of autonomous driving. Leveraging a newly designed control transfer mechanism between human and automation, human is able to intervene and correct the agent's unreasonable actions in real time when necessary during the model training process. Based on this human-in-the-loop guidance mechanism, an improved actor-critic architecture with modified policy and value networks is developed. The fast convergence of the proposed Hug-DRL allows real-time human guidance actions to be fused into the agent's training loop, further improving the efficiency and performance of deep reinforcement learning. The developed method is validated by human-in-the-loop experiments with 40 subjects and compared with other state-of-the-art learning approaches. The results suggest that the proposed method can effectively enhance the training efficiency and performance of the deep reinforcement learning algorithm under human guidance, without imposing specific requirements on participant expertise and experience.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 21 of its 48 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

21 of 48 references.

References · 21

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  5. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  6. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 369
  7. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  8. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  9. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  10. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  11. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 224
  12. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
9 more
  1. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  2. 2018Count-Based Exploration with the Successor Representationcited by 112
  3. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 102
  4. 2019Hindsight Credit Assignmentcited by 74
  5. 2018A Reinforcement Learning Based Approach for Automated Lane Change Maneuverscited by 44
  6. 2020Multi-modal Sensor Fusion-Based Deep Neural Network for End-to-end Autonomous Driving with Scene Understandingcited by 42
  7. 2020High-speed Autonomous Drifting with Deep Reinforcement Learningcited by 20
  8. 2020Neuro-symbolic Architectures for Context Understandingcited by 13
  9. 2018Intervention Aided Reinforcement Learning for Safe and Practical Policy Optimization in Navigationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 7; 2023: 1; 2024: 2; 2025: 2; 2026: 2.

16 papers.

Sort:

By area:

  1. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimizationcited by 44
  2. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  3. 2021Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancementcited by 28
  4. 2022Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Drivingcited by 24
  5. 2023Decision Making for Human-in-the-loop Robotic Agents via Uncertainty-Aware Reinforcement Learningcited by 9
  6. 2025Knowledge Augmented Complex Problem Solving with Large Language Models: A Surveycited by 6
  7. 2022A-Eye: Driving with the Eyes of AI for Corner Case Generationcited by 5
  8. 2022Safe Decision-making for Lane-change of Autonomous Vehicles via Human Demonstration-aided Reinforcement Learningcited by 5
  9. 2022Human-AI Shared Control via Policy Dissectioncited by 4
  10. 2024Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimizationcited by 4
  11. 2021Uncertainty-Aware Model-Based Reinforcement Learning with Application to Autonomous Drivingcited by 3
  12. 2022Exploring the trade off between human driving imitation and safety for traffic simulationcited by 3
4 more
  1. 2022Sampling Efficient Deep Reinforcement Learning through Preference-Guided Stochastic Explorationcited by 2
  2. 2025Interactive Double Deep Q-network: Integrating Human Interventions and Evaluative Predictions in Reinforcement Learning of Autonomous Drivingcited by 0
  3. 2026Bridging Policy and Real-World Dynamics: LLM-Augmented Rebalancing for Shared Micromobility Systemscited by 0
  4. 2026Themis: An explainable AI-enabled framework for Reinforcement Learning with Human Feedbackcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving
  2. 2025From Learning to Mastery: Achieving Safe and Efficient Real-World Autonomous Driving with Human-In-The-Loop Reinforcement Learning
  3. 2021Modified DDPG car-following model with a real-world human driving experience with CARLA simulator
  4. 2024Adaptive Autopilot: Constrained DRL for Diverse Driving Behaviors
  5. 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving
  6. 2023Review of Deep Reinforcement Learning for Autonomous Driving
  7. 2020Deep Reinforcement Learning for Autonomous Driving: A Survey
  8. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learning