mlsanityData to 14 Oct 2026

A Systematic Approach to Design Real-World Human-in-the-Loop Deep Reinforcement Learning: Salient Features, Challenges and Trade-offs

arXiv 2504.17006 ↗First posted 23 Apr 2025cs.AI cs.LG cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

With the growing popularity of deep reinforcement learning (DRL), human-in-the-loop (HITL) approach has the potential to revolutionize the way we approach decision-making problems and create new opportunities for human-AI collaboration. In this article, we introduce a novel multi-layered hierarchical HITL DRL algorithm that comprises three types of learning: self learning, imitation learning and transfer learning. In addition, we consider three forms of human inputs: reward, action and demonstration. Furthermore, we discuss main challenges, trade-offs and advantages of HITL in solving complex problems and how human information can be integrated in the AI solution systematically. To verify our technical results, we present a real-world unmanned aerial vehicles (UAV) problem wherein a number of enemy drones attack a restricted area. The objective is to design a scalable HITL DRL algorithm for ally drones to neutralize the enemy drones before they reach the area. To this end, we first implement our solution using an award-winning open-source HITL software called Cogment. We then demonstrate several interesting results such as (a) HITL leads to faster training and higher performance, (b) advice acts as a guiding direction for gradient methods and lowers variance, and (c) the amount of advice should neither be too large nor too small to avoid over-training and under-training. Finally, we illustrate the role of human-AI cooperation in solving two real-world complex scenarios, i.e., overloaded and decoy attacks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 6 of its 20 references that are in the map. First posted on arXiv 23 Apr 2025.

Builds on

6 of 20 references.

References · 6

  1. 2017Deep reinforcement learning from human preferencescited by 3,912
  2. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  3. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  4. 2017Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spacescited by 123
  5. 2020High-speed Autonomous Drifting with Deep Reinforcement Learningcited by 24
  6. 2021Cogment: Open Source Framework For Distributed Multi-actor Training, Deployment & Operationscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunities
  2. 2021Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous Driving
  3. 2026Deep Reinforcement Learning: From First Principles to Reasoning Models ↗
  4. 2024A Digital Twin Framework for Reinforcement Learning with Real-Time Self-Improvement via Human Assistive Teleoperation
  5. 2023A Survey of Reinforcement Learning from Human Feedback
  6. 2024HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving
  7. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes
  8. 2024Generative AI for Deep Reinforcement Learning: Framework, Analysis, and Use Cases