# Leveraging Human Guidance for Deep Reinforcement Learning Tasks

arXiv: https://arxiv.org/abs/1909.09906 · HTML: https://mlsanity.com/paper/1909.09906/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2019.

## Built on by (most cited first, 30 of 42)

- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [Human-in-the-Loop Imitation Learning using Remote Teleoperation](https://mlsanity.com/paper/2012.06733/) (2020)
- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction](https://mlsanity.com/paper/2405.10315/) (2024)
- [The EMPATHIC Framework for Task Learning from Implicit Human Feedback](https://mlsanity.com/paper/2009.13649/) (2020)
- [A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning](https://mlsanity.com/paper/2210.04561/) (2022)
- [Task Success is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviors](https://mlsanity.com/paper/2402.04210/) (2024)
- [Leveraging Approximate Symbolic Models for Reinforcement Learning via Skill Diversity](https://mlsanity.com/paper/2202.02886/) (2022)
- [Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback](https://mlsanity.com/paper/2402.02423/) (2024)
- [Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptation](https://mlsanity.com/paper/2307.06333/) (2023)
- [Primitive Skill-based Robot Learning from Human Evaluative Feedback](https://mlsanity.com/paper/2307.15801/) (2023)
- [Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks](https://mlsanity.com/paper/2107.05825/) (2021)
- [NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities](https://mlsanity.com/paper/2311.01454/) (2023)
- [Relative Behavioral Attributes: Filling the Gap between Symbolic Goal Specification and Reward Learning from Human Preferences](https://mlsanity.com/paper/2210.15906/) (2022)
- [Efficiently Guiding Imitation Learning Agents with Human Gaze](https://mlsanity.com/paper/2002.12500/) (2020)
- [Selective Exploration and Information Gathering in Search and Rescue Using Hierarchical Learning Guided by Natural Language Input](https://mlsanity.com/paper/2409.13445/) (2024)
- [Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework](https://mlsanity.com/paper/2411.11761/) (2024)
- [Accelerating Reinforcement Learning Agent with EEG-based Implicit Human Feedback](https://mlsanity.com/paper/2006.16498/) (2020)
- [DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation](https://mlsanity.com/paper/2104.00163/) (2021)
- [Deception in Social Learning: A Multi-Agent Reinforcement Learning Perspective](https://mlsanity.com/paper/2106.05402/) (2021)
- [Human-AI Shared Control via Policy Dissection](https://mlsanity.com/paper/2206.00152/) (2022)
- [DIAMBRA Arena: a New Reinforcement Learning Platform for Research and Experimentation](https://mlsanity.com/paper/2210.10595/) (2022)
- [Cogment: Open Source Framework For Distributed Multi-actor Training, Deployment & Operations](https://mlsanity.com/paper/2106.11345/) (2021)
- [Good Time to Ask: A Learning Framework for Asking for Help in Embodied Visual Navigation](https://mlsanity.com/paper/2206.10606/) (2022)
- [A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges](https://mlsanity.com/paper/2211.06665/) (2022)
- [Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement](https://mlsanity.com/paper/2606.18247/) (2026)
- [Towards Interactive Reinforcement Learning with Intrinsic Feedback](https://mlsanity.com/paper/2112.01575/) (2021)
- [Teachable Reinforcement Learning via Advice Distillation](https://mlsanity.com/paper/2203.11197/) (2022)
- [Scalable Interactive Machine Learning for Future Command and Control](https://mlsanity.com/paper/2402.06501/) (2024)

## Abstract (arXiv)

Reinforcement learning agents can learn to solve sequential decision tasks by interacting with the environment. Human knowledge of how to solve these tasks can be incorporated using imitation learning, where the agent learns to imitate human demonstrated decisions. However, human guidance is not limited to the demonstrations. Other types of guidance could be more suitable for certain tasks and require less human effort. This survey provides a high-level overview of five recent learning frameworks that primarily rely on human guidance other than conventional, step-by-step action demonstrations. We review the motivation, assumption, and implementation of each framework. We then discuss possible future research directions.