# Control of Memory, Active Perception, and Action in Minecraft

arXiv: https://arxiv.org/abs/1605.09128 · HTML: https://mlsanity.com/paper/1605.09128/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2016.

## Built on by (most cited first, 30 of 61)

- [Offline Reinforcement Learning as One Big Sequence Modeling Problem](https://mlsanity.com/paper/2106.02039/) (2021)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos](https://mlsanity.com/paper/2206.11795/) (2022)
- [Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs](https://mlsanity.com/paper/2110.05038/) (2021)
- [Structured Scene Memory for Vision-Language Navigation](https://mlsanity.com/paper/2103.03454/) (2021)
- [AndroidEnv: A Reinforcement Learning Platform for Android](https://mlsanity.com/paper/2105.13231/) (2021)
- [Deep Reinforcement Learning from Policy-Dependent Human Feedback](https://mlsanity.com/paper/1902.04257/) (2019)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Scene-Intuitive Agent for Remote Embodied Visual Grounding](https://mlsanity.com/paper/2103.12944/) (2021)
- [The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors](https://mlsanity.com/paper/2101.11071/) (2021)
- [Local-Global Context Aware Transformer for Language-Guided Video Segmentation](https://mlsanity.com/paper/2203.09773/) (2022)
- [Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning](https://mlsanity.com/paper/2502.10550/) (2025)
- [Online Adaptation of Language Models with a Memory of Amortized Contexts](https://mlsanity.com/paper/2403.04317/) (2024)
- [MCU: An Evaluation Framework for Open-Ended Game Agents](https://mlsanity.com/paper/2310.08367/) (2023)
- [ReflAct: World-Grounded Decision Making in LLM Agents via Goal-State Reflection](https://mlsanity.com/paper/2505.15182/) (2025)
- [JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2112.04907/) (2021)
- [Synthetic Returns for Long-Term Credit Assignment](https://mlsanity.com/paper/2102.12425/) (2021)
- [Large-Scale Retrieval for Reinforcement Learning](https://mlsanity.com/paper/2206.05314/) (2022)
- [Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learning](https://mlsanity.com/paper/2205.14850/) (2022)
- [Object Memory Transformer for Object Goal Navigation](https://mlsanity.com/paper/2203.14708/) (2022)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Teaching Agents how to Map: Spatial Reasoning for Multi-Object Navigation](https://mlsanity.com/paper/2107.06011/) (2021)
- [Learning Reward Machines: A Study in Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2112.09477/) (2021)
- [Navigating to Objects in Unseen Environments by Distance Prediction](https://mlsanity.com/paper/2202.03735/) (2022)
- [Light-weight probing of unsupervised representations for Reinforcement Learning](https://mlsanity.com/paper/2208.12345/) (2022)
- [Learning from Good Trajectories in Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2211.15612/) (2022)
- [LEGENT: Open Platform for Embodied Agents](https://mlsanity.com/paper/2404.18243/) (2024)
- [MaAST: Map Attention with Semantic Transformersfor Efficient Visual Navigation](https://mlsanity.com/paper/2103.11374/) (2021)
- [Explain My Surprise: Learning Efficient Long-Term Memory by Predicting Uncertain Outcomes](https://mlsanity.com/paper/2207.13649/) (2022)
- [ActiView: Evaluating Active Perception Ability for Multimodal Large Language Models](https://mlsanity.com/paper/2410.04659/) (2024)

## Abstract (arXiv)

In this paper, we introduce a new set of reinforcement learning (RL) tasks in Minecraft (a flexible 3D world). We then use these tasks to systematically compare and contrast existing deep reinforcement learning (DRL) architectures with our new memory-based DRL architectures. These tasks are designed to emphasize, in a controllable manner, issues that pose challenges for RL methods including partial observability (due to first-person visual observations), delayed rewards, high-dimensional visual observations, and the need to use active perception in a correct manner so as to perform well in the tasks. While these tasks are conceptually simple to describe, by virtue of having all of these challenges simultaneously they are difficult for current DRL architectures. Additionally, we evaluate the generalization performance of the architectures on environments not used during training. The experimental results show that our new architectures generalize to unseen environments better than existing DRL architectures.