mlsanityData to 7 Oct 2026

MineRL: A Large-Scale Dataset of Minecraft Demonstrations

arXiv 1907.13440 ↗First posted 29 Jul 2019cs.LG cs.AI cs.NEcited by 210

Not on the map. Why?

Cited by 45 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The sample inefficiency of standard deep reinforcement learning methods precludes their application to many real-world problems. Methods which leverage human demonstrations require fewer samples but have been researched less. As demonstrated in the computer vision and natural language processing communities, large-scale datasets have the capacity to facilitate research by serving as an experimental and benchmarking platform for new methods. However, existing datasets compatible with reinforcement learning simulators do not have sufficient scale, structure, and quality to enable the further development and evaluation of methods focused on using human examples. Therefore, we introduce a comprehensive, large-scale, simulator-paired dataset of human demonstrations: MineRL. The dataset consists of over 60 million automatically annotated state-action pairs across a variety of related tasks in Minecraft, a dynamic, 3D, open-world environment. We present a novel data collection scheme which allows for the ongoing introduction of new tasks and the gathering of complete state information suitable for a variety of methods. We demonstrate the hierarchality, diversity, and scale of the MineRL dataset. Further, we show the difficulty of the Minecraft domain along with the potential of MineRL in developing techniques to solve key research challenges within it.

Key facts. Cited by 210 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 3; 2021: 18; 2022: 25; 2023: 40; 2024: 43; 2025: 49; 2026: 32.

210 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  2. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  3. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  4. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  5. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  6. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  7. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  8. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  9. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 170
  10. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  11. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  12. 2023Language Models Meet World Models: Embodied Experiences Enhance Language Modelscited by 123
38 more
  1. 2024Cradle: Empowering Foundation Agents Towards General Computer Controlcited by 114
  2. 2024VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agentscited by 101
  3. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 95
  4. 2021Learning to See by Looking at Noisecited by 92
  5. 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
  6. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  7. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 83
  8. 2024Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Taskscited by 83
  9. 2023MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perceptioncited by 67
  10. 2024Scaling Instructable Agents Across Many Simulated Worldscited by 67
  11. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 60
  12. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 56
  13. 2023See and Think: Embodied Agent in Virtual Environmentcited by 54
  14. 2024MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Controlcited by 52
  15. 2021Counter-Strike Deathmatch with Large-Scale Behavioural Cloningcited by 49
  16. 2024Project Sid: Many-agent simulations toward AI civilizationcited by 46
  17. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 46
  18. 2023Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worldscited by 45
  19. 2022Temporally Consistent Transformers for Video Generationcited by 43
  20. 2023GROOT: Learning to Follow Instructions by Watching Gameplay Videoscited by 42
  21. 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agentscited by 41
  22. 2021The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priorscited by 40
  23. 2021Clockwork Variational Autoencoderscited by 40
  24. 2023Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Predictioncited by 39
  25. 2026Solaris: Building a Multiplayer Video World Model in Minecraftcited by 38
  26. 2025VRAG: Learning World Models for Interactive Video Generationcited by 37
  27. 2023Skill Reinforcement Learning and Planning for Open-World Long-Horizon Taskscited by 35
  28. 2021Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraftcited by 34
  29. 2023MCU: An Evaluation Framework for Open-Ended Game Agentscited by 33
  30. 2024RL-GPT: Integrating Reinforcement Learning and Code-as-policycited by 32
  31. 2021The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactionscited by 31
  32. 2022Learning to Execute Actions or Ask Clarification Questionscited by 30
  33. 2024OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agentscited by 30
  34. 2023Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Gamescited by 29
  35. 2021JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learningcited by 28
  36. 2022Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learningcited by 27
  37. 2023RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalizationcited by 27
  38. 2025JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mousecited by 27

50 most cited of 210 All 210 (JSON)