# Improving Deep Reinforcement Learning in Minecraft with Action Advice

arXiv: https://arxiv.org/abs/1908.01007 · HTML: https://mlsanity.com/paper/1908.01007/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Aug 2019.

## Built on by (most cited first, 10 of 10)

- [AndroidEnv: A Reinforcement Learning Platform for Android](https://mlsanity.com/paper/2105.13231/) (2021)
- [LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence](https://mlsanity.com/paper/2405.17424/) (2024)
- [Interactive Machine Learning: A State of the Art Review](https://mlsanity.com/paper/2207.06196/) (2022)
- [Swapped goal-conditioned offline reinforcement learning](https://mlsanity.com/paper/2302.08865/) (2023)
- [Human-AI Collaboration in Real-World Complex Environment with Reinforcement Learning](https://mlsanity.com/paper/2312.15160/) (2023)
- [Revisiting Estimation Bias in Policy Gradients for Deep Reinforcement Learning](https://mlsanity.com/paper/2301.08442/) (2023)
- [Lateralized Learning for Multi-Class Visual Classification Tasks](https://mlsanity.com/paper/2301.12637/) (2023)
- [Towards Goal-Oriented Agents for Evolving Problems Observed via Conversation](https://mlsanity.com/paper/2401.05822/) (2024)
- [Prioritized offline Goal-swapping Experience Replay](https://mlsanity.com/paper/2302.07741/) (2023)
- [Boosting Robustness in Preference-Based Reinforcement Learning with Dynamic Sparsity](https://mlsanity.com/paper/2406.06495/) (2024)

## Abstract (arXiv)

Training deep reinforcement learning agents complex behaviors in 3D virtual environments requires significant computational resources. This is especially true in environments with high degrees of aliasing, where many states share nearly identical visual features. Minecraft is an exemplar of such an environment. We hypothesize that interactive machine learning IML, wherein human teachers play a direct role in training through demonstrations, critique, or action advice, may alleviate agent susceptibility to aliasing. However, interactive machine learning is only practical when the number of human interactions is limited, requiring a balance between human teacher effort and agent performance. We conduct experiments with two reinforcement learning algorithms which enable human teachers to give action advice, Feedback Arbitration and Newtonian Action Advice, under visual aliasing conditions. To assess potential cognitive load per advice type, we vary the accuracy and frequency of various human action advice techniques. Training efficiency, robustness against infrequent and inaccurate advisor input, and sensitivity to aliasing are examined.