mlsanityData to 14 Oct 2026

JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learning

arXiv 2112.04907 ↗First posted 7 Dec 2021cs.LG cs.AIcited by 28

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Learning rational behaviors in open-world games like Minecraft remains to be challenging for Reinforcement Learning (RL) research due to the compound challenge of partial observability, high-dimensional visual perception and delayed reward. To address this, we propose JueWu-MC, a sample-efficient hierarchical RL approach equipped with representation learning and imitation learning to deal with perception and exploration. Specifically, our approach includes two levels of hierarchy, where the high-level controller learns a policy to control over options and the low-level workers learn to solve each sub-task. To boost the learning of sub-tasks, we propose a combination of techniques including 1) action-aware representation learning which captures underlying relations between action and representation, 2) discriminator-based self-imitation learning for efficient exploration, and 3) ensemble behavior cloning with consistency filtering for policy robustness. Extensive experiments show that JueWu-MC significantly improves sample efficiency and outperforms a set of baselines by a large margin. Notably, we won the championship of the NeurIPS MineRL 2021 research competition and achieved the highest performance score ever.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 32 of its 43 references that are in the map. First posted on arXiv 7 Dec 2021.

Builds on

32 of 43 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021SEIHAI: A Sample-efficient Hierarchical AI for the MineRL Competition17 Nov
  2. 2020Scaling Imitation Learning in Minecraft6 Jul
  3. 2019Imitation Learning from Imperfect Demonstration27 Jan
  4. 2018Self-Imitation Learning14 Jun

Other references · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  6. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  7. 2015Prioritized Experience Replaycited by 1,099
  8. 2018Exploration by Random Network Distillationcited by 993
  9. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  10. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  11. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  12. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 340
16 more
  1. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
  2. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  3. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
  4. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 161
  5. 2018State Representation Learning for Control: An Overviewcited by 127
  6. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 98
  7. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  8. 2018Hierarchical Imitation and Reinforcement Learningcited by 88
  9. 2016Control of Memory, Active Perception, and Action in Minecraftcited by 61
  10. 2021The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priorscited by 41
  11. 2019Dynamics-aware Embeddingscited by 34
  12. 2020Supervised Learning Achieves Human-Level Performance in MOBA Games: A Case Study of Honor of Kingscited by 18
  13. 2021TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrationscited by 18
  14. 2020Sample Efficient Reinforcement Learning through Learning from Demonstrations in Minecraftcited by 15
  15. 2020Which Heroes to Pick? Learning to Draft in MOBA Games with Neural Networks and Tree Searchcited by 10
  16. 2020Playing Minecraft with Behavioural Cloningcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 13; 2024: 5; 2025: 4; 2026: 2.

28 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  2. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 398
  3. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  4. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 289
  5. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 171
  6. 2023Language Models Meet World Models: Embodied Experiences Enhance Language Modelscited by 124
  7. 2023MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perceptioncited by 67
  8. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 61
  9. 2023See and Think: Embodied Agent in Virtual Environmentcited by 54
  10. 2023GROOT: Learning to Follow Instructions by Watching Gameplay Videoscited by 42
  11. 2023Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Predictioncited by 39
  12. 2023TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systemscited by 39
16 more
  1. 2022MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learnedcited by 26
  2. 2024Odyssey: Empowering Minecraft Agents with Open-World Skillscited by 26
  3. 2024Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigationcited by 24
  4. 2024S-Agents: Self-organizing Agents in Open-ended Environmentscited by 22
  5. 2022Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualizationcited by 14
  6. 2023Reinforcement Learning Friendly Vision-Language Model for Minecraftcited by 14
  7. 2025Position: Interactive Generative Video as Next-Generation Game Enginecited by 13
  8. 2026UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experiencecited by 11
  9. 2024Affordable Generative Agentscited by 9
  10. 2024MrSteve: Instruction-Following Agents in Minecraft with What-Where-When Memorycited by 9
  11. 2022RLogist: Fast Observation Strategy on Whole-slide Images with Deep Reinforcement Learningcited by 8
  12. 2025EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Abilitycited by 6
  13. 2023Sample Dropout: A Simple yet Effective Variance Reduction Technique in Deep Policy Optimizationcited by 3
  14. 2025Select before Act: Spatially Decoupled Action Repetition for Continuous Controlcited by 0
  15. 2025Goal-Guided Efficient Exploration via Large Language Model in Reinforcement Learningcited by 0
  16. 2026Advantage-based Temporal Attack in Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Zero-shot Imitation Policy via Search in Demonstration Dataset
  2. 2023Behavioral Cloning via Search in Embedded Demonstration Dataset
  3. 2025Training Agents Inside of Scalable World Models
  4. 2022Behavioral Cloning via Search in Video PreTraining Latent Space
  5. 2021Combining Learning from Human Feedback and Knowledge Engineering to Solve Hierarchical Tasks in Minecraft
  6. 2025Optimus-2: Multimodal Minecraft Agent with Goal-Observation-Action Conditioned Policy
  7. 2020Retrospective Analysis of the 2019 MineRL Competition on Sample Efficient Reinforcement Learning
  8. 2026Unsupervised Hierarchical Skill Discovery