# Improving Reinforcement Learning with Human Assistance: An Argument for Human Subject Studies with HIPPO Gym

arXiv: https://arxiv.org/abs/2102.02639 · HTML: https://mlsanity.com/paper/2102.02639/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 63 references that are in the map.
- First posted on arXiv 2 Feb 2021.

## Builds on (most cited first, 15 of 15)

- [Interactive Learning from Policy-Dependent Human Feedback](https://mlsanity.com/paper/1701.06049/) (2017)
- [A Survey of Reinforcement Learning Informed by Natural Language](https://mlsanity.com/paper/1906.03926/) (2019)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [Explainability in Deep Reinforcement Learning](https://mlsanity.com/paper/2008.06693/) (2020)
- [Benchmarking Reinforcement Learning Algorithms on Real-World Robots](https://mlsanity.com/paper/1809.07731/) (2018)
- [Learning to Teach in Cooperative Multiagent Reinforcement Learning](https://mlsanity.com/paper/1805.07830/) (2018)
- [The EMPATHIC Framework for Task Learning from Implicit Human Feedback](https://mlsanity.com/paper/2009.13649/) (2020)
- [A Conceptual Framework for Externally-influenced Agents: An Assisted Reinforcement Learning Review](https://mlsanity.com/paper/2007.01544/) (2020)
- [Learning to Teach Reinforcement Learning Agents](https://mlsanity.com/paper/1707.09079/) (2017)
- [Pre-training with Non-expert Human Demonstration for Deep Reinforcement Learning](https://mlsanity.com/paper/1812.08904/) (2018)
- [Leveraging human knowledge in tabular reinforcement learning: A study of human subjects](https://mlsanity.com/paper/1805.05769/) (2018)
- [Efficiently Guiding Imitation Learning Agents with Human Gaze](https://mlsanity.com/paper/2002.12500/) (2020)
- [Theoretically-Grounded Policy Advice from Multiple Teachers in Reinforcement Learning Settings with Applications to Negative Transfer](https://mlsanity.com/paper/1604.03986/) (2016)
- [Useful Policy Invariant Shaping from Arbitrary Advice](https://mlsanity.com/paper/2011.01297/) (2020)
- [A Human-Centered Approach to Interactive Machine Learning](https://mlsanity.com/paper/1905.06289/) (2019)

## Built on by (most cited first, 3 of 3)

- [RLHF-Blender: A Configurable Interactive Interface for Learning from Diverse Human Feedback](https://mlsanity.com/paper/2308.04332/) (2023)
- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [Can You Improve My Code? Optimizing Programs with Local Search](https://mlsanity.com/paper/2307.05603/) (2023)

## Abstract (arXiv)

Reinforcement learning (RL) is a popular machine learning paradigm for game playing, robotics control, and other sequential decision tasks. However, RL agents often have long learning times with high data requirements because they begin by acting randomly. In order to better learn in complex tasks, this article argues that an external teacher can often significantly help the RL agent learn. OpenAI Gym is a common framework for RL research, including a large number of standard environments and agents, making RL research significantly more accessible. This article introduces our new open-source RL framework, the Human Input Parsing Platform for Openai Gym (HIPPO Gym), and the design decisions that went into its creation. The goal of this platform is to facilitate human-RL research, again lowering the bar so that more researchers can quickly investigate different ways that human teachers could assist RL agents, including learning from demonstrations, learning from feedback, or curriculum learning.