mlsanityData to 14 Oct 2026

GUIDE: Real-Time Human-Shaped Agents

arXiv 2410.15181 ↗First posted 19 Oct 2024cs.LG cs.HCcited by 3

Not on the map. Why?

Abstract (arXiv)

The recent rapid advancement of machine learning has been driven by increasingly powerful models with the growing availability of training data and computational resources. However, real-time decision-making tasks with limited time and sparse learning signals remain challenging. One way of improving the learning speed and performance of these agents is to leverage human guidance. In this work, we introduce GUIDE, a framework for real-time human-guided reinforcement learning by enabling continuous human feedback and grounding such feedback into dense rewards to accelerate policy learning. Additionally, our method features a simulated feedback module that learns and replicates human feedback patterns in an online fashion, effectively reducing the need for human input while allowing continual training. We demonstrate the performance of our framework on challenging tasks with sparse rewards and visual observations. Our human study involving 50 subjects offers strong quantitative and qualitative evidence of the effectiveness of our approach. With only 10 minutes of human feedback, our algorithm achieves up to 30% increase in success rate compared to its RL baseline.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 22 of its 53 references that are in the map. First posted on arXiv 19 Oct 2024.

Builds on

22 of 53 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2017Deep reinforcement learning from human preferencescited by 3,912
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  6. 2016End to End Learning for Self-Driving Carscited by 1,009
  7. 2018Exploration by Random Network Distillationcited by 993
  8. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  9. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  10. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  11. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
  12. 2018Reward learning from human preferences and demonstrations in Ataricited by 316
10 more
  1. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  2. 2017Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spacescited by 123
  3. 2019Deep Reinforcement Learning from Policy-Dependent Human Feedbackcited by 56
  4. 2018Practical Deep Reinforcement Learning Approach for Stock Tradingcited by 49
  5. 2018DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedbackcited by 41
  6. 2019The problem with DDPG: understanding failures in deterministic environments with sparse rewardscited by 20
  7. 2019Visual Hide and Seekcited by 13
  8. 2024CREW: Facilitating Human-AI Teaming Researchcited by 11
  9. 2021Visual Perspective Taking for Opponent Behavior Modelingcited by 6
  10. 2022Challenges to Solving Combinatorially Hard Long-Horizon Deep RL Taskscited by 4

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025CREW-WILDFIRE: Benchmarking Agentic Multi-Agent Collaborations at Scalecited by 7
  2. 2025The Term 'Agent' Has Been Diluted Beyond Utility and Requires Redefinitioncited by 6
  3. 2025Pref-GUIDE: Continual Policy Learning from Real-Time Human Feedback via Preference-Based Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework
  2. 2019Leveraging Human Guidance for Deep Reinforcement Learning Tasks
  3. 2025Reinforcement Learning from Human Feedback
  4. 2023Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedback
  5. 2023A Survey of Reinforcement Learning from Human Feedback
  6. 2024Unveiling the Role of Expert Guidance: A Comparative Analysis of User-centered Imitation Learning and Traditional Reinforcement Learning
  7. 2025Reinforcement Learning from Multi-level and Episodic Human Feedback
  8. 2020Reward Shaping for Human Learning via Inverse Reinforcement Learning