# Newtonian Action Advice: Integrating Human Verbal Instruction with Reinforcement Learning

arXiv: https://arxiv.org/abs/1804.05821 · HTML: https://mlsanity.com/paper/1804.05821/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2018.

## Built on by (most cited first, 3 of 3)

- [Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework](https://mlsanity.com/paper/2411.11761/) (2024)
- [Lateralization in Agents' Decision Making: Evidence of Benefits/Costs from Artificial Intelligence](https://mlsanity.com/paper/2302.01542/) (2023)
- [Prosody as a Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic Implications](https://mlsanity.com/paper/2410.23554/) (2024)

## Abstract (arXiv)

A goal of Interactive Machine Learning (IML) is to enable people without specialized training to teach agents how to perform tasks. Many of the existing machine learning algorithms that learn from human instructions are evaluated using simulated feedback and focus on how quickly the agent learns. While this is valuable information, it ignores important aspects of the human-agent interaction such as frustration. In this paper, we present the Newtonian Action Advice agent, a new method of incorporating human verbal action advice with Reinforcement Learning (RL) in a way that improves the human-agent interaction. In addition to simulations, we validated the Newtonian Action Advice algorithm by conducting a human-subject experiment. The results show that Newtonian Action Advice can perform better than Policy Shaping, a state-of-the-art IML algorithm, both in terms of RL metrics like cumulative reward and human factors metrics like frustration.