# Reinforcement learning with human advice: a survey

arXiv: https://arxiv.org/abs/2005.11016 · HTML: https://mlsanity.com/paper/2005.11016/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2020.

## Built on by (most cited first, 24 of 24)

- [A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future](https://mlsanity.com/paper/2504.12328/) (2025)
- [Prompt Engineering for Healthcare: Methodologies and Applications](https://mlsanity.com/paper/2304.14670/) (2023)
- [Perspectives on the Social Impacts of Reinforcement Learning with Human Feedback](https://mlsanity.com/paper/2303.02891/) (2023)
- [Decision Making for Human-in-the-loop Robotic Agents via Uncertainty-Aware Reinforcement Learning](https://mlsanity.com/paper/2303.06710/) (2023)
- [Hybrid Approaches for Moral Value Alignment in AI Agents: a Manifesto](https://mlsanity.com/paper/2312.01818/) (2023)
- [Towards Human-Like RL: Taming Non-Naturalistic Behavior in Deep RL via Adaptive Behavioral Costs in 3D Games](https://mlsanity.com/paper/2309.15484/) (2023)
- [Interactive Imitation Learning for Dexterous Robotic Manipulation: Challenges and Perspectives -- A Survey](https://mlsanity.com/paper/2506.00098/) (2025)
- [Good Time to Ask: A Learning Framework for Asking for Help in Embodied Visual Navigation](https://mlsanity.com/paper/2206.10606/) (2022)
- [CROP: Towards Distributional-Shift Robust Reinforcement Learning using Compact Reshaped Observation Processing](https://mlsanity.com/paper/2304.13616/) (2023)
- [On Bits and Bandits: Quantifying the Regret-Information Trade-off](https://mlsanity.com/paper/2405.16581/) (2024)
- [Opinion-Guided Reinforcement Learning](https://mlsanity.com/paper/2405.17287/) (2024)
- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [Towards Interactive Reinforcement Learning with Intrinsic Feedback](https://mlsanity.com/paper/2112.01575/) (2021)
- [Complex Model Transformations by Reinforcement Learning with Uncertain Human Guidance](https://mlsanity.com/paper/2506.20883/) (2025)
- [Interactive Program Synthesis for Modeling Collaborative Physical Activities from Narrated Demonstrations](https://mlsanity.com/paper/2509.24250/) (2025)
- [Modeling Human Behavior Part II -- Cognitive approaches and Uncertainty](https://mlsanity.com/paper/2205.06483/) (2022)
- [Modeling Human Behavior Part I -- Learning and Belief Approaches](https://mlsanity.com/paper/2205.06485/) (2022)
- [DAPPER: Discriminability-Aware Policy-to-Policy Preference-Based Reinforcement Learning for Query-Efficient Robot Skill Acquisition](https://mlsanity.com/paper/2505.06357/) (2025)
- [Contextualizing Artificially Intelligent Morality: A Meta-Ethnography of Top-Down, Bottom-Up, and Hybrid Models for Theoretical and Applied Ethics in Artificial Intelligence](https://mlsanity.com/paper/2204.07612/) (2022)
- [Optimal Transport for Correctional Learning](https://mlsanity.com/paper/2304.01701/) (2023)
- [Adaptive Querying for Reward Learning from Human Feedback](https://mlsanity.com/paper/2412.07990/) (2024)
- [Robots and Children that Learn Together : Improving Knowledge Retention by Teaching Peer-Like Interactive Robots](https://mlsanity.com/paper/2506.18365/) (2025)
- [Feedback Manipulation Regularization: Enabling Offline Agent Alignment for Imitation Learning](https://mlsanity.com/paper/2607.07859/) (2026)
- [Inference-Time Policy Alignment for Fair Reinforcement Learning](https://mlsanity.com/paper/2608.00175/) (2026)

## Abstract (arXiv)

In this paper, we provide an overview of the existing methods for integrating human advice into a Reinforcement Learning process. We first propose a taxonomy of the different forms of advice that can be provided to a learning agent. We then describe the methods that can be used for interpreting advice when its meaning is not determined beforehand. Finally, we review different approaches for integrating advice into the learning process.