mlsanityData to 7 Oct 2026

Dungeons and Data: A Large-Scale NetHack Dataset

arXiv 2211.00539 ↗First posted 1 Nov 2022cs.LG cs.AIcited by 16

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Recent breakthroughs in the development of agents to solve challenging sequential decision making problems such as Go, StarCraft, or DOTA, have relied on both simulated environments and large-scale datasets. However, progress on this research has been hindered by the scarcity of open-sourced datasets and the prohibitive computational cost to work with them. Here we present the NetHack Learning Dataset (NLD), a large and highly-scalable dataset of trajectories from the popular game of NetHack, which is both extremely challenging for current methods and very fast to run. NLD consists of three parts: 10 billion state transitions from 1.5 million human trajectories collected on the NAO public NetHack server from 2009 to 2020; 3 billion state-action-score transitions from 100,000 trajectories collected from the symbolic bot winner of the NetHack Challenge 2021; and, accompanying code for users to record, load and stream any collection of such trajectories in a highly compressed form. We evaluate a wide range of existing algorithms including online and offline RL, as well as learning from demonstrations, showing that significant research advances are needed to fully leverage large-scale datasets for challenging sequential decision making tasks.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 41 of its 90 references that are in the map. First posted on arXiv 1 Nov 2022.

Builds on

41 of 90 references.

Closest ancestors · 3

Same direction, newest first

  1. 2020The MAGICAL Benchmark for Robust Imitation1 Nov
  2. 2018Imitating Latent Policies from Observation21 May
  3. 2018Behavioral Cloning from Observation4 May

Other references · 38

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  8. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  9. 2019Solving Rubik's Cube with a Robot Handcited by 699
  10. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  11. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  12. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
26 more
  1. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  2. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  3. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  4. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  5. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  6. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  7. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
  8. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
  9. 2020The NetHack Learning Environmentcited by 196
  10. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 159
  11. 2022Adversarially Trained Actor Critic for Offline Reinforcement Learningcited by 111
  12. 2017Third-Person Imitation Learningcited by 103
  13. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 96
  14. 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 89
  15. 2020Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learningcited by 84
  16. 2018Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitationcited by 76
  17. 2018Kickstarting Deep Reinforcement Learningcited by 75
  18. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learningcited by 63
  19. 2022You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environmentscited by 62
  20. 2019An Optimistic Perspective on Offline Reinforcement Learningcited by 37
  21. 2021RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learningcited by 31
  22. 2022Insights From the NeurIPS 2021 NetHack Challengecited by 20
  23. 2019Scaling data-driven robotics with reward sketching and batch reinforcement learningcited by 18
  24. 2021BRAC+: Improved Behavior Regularized Actor Critic for Offline Reinforcement Learningcited by 13
  25. 2019Simitate: A Hybrid Imitation Learning Benchmarkcited by 10
  26. 2022Hierarchical Kickstarting for Skill Transfer in Reinforcement Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 4; 2025: 2; 2026: 1.

16 papers.

Sort:

By area:

  1. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  2. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99
  3. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 58
  4. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 40
  5. 2023Emergence of In-Context Reinforcement Learning from Noise Distillationcited by 26
  6. 2023The Generalization Gap in Offline Reinforcement Learningcited by 25
  7. 2024MaestroMotif: Skill Design from Artificial Intelligence Feedbackcited by 24
  8. 2024Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problemcited by 23
  9. 2025ProgRM: Build Better GUI Agents with Progress Rewardscited by 21
  10. 2023NetHack is Hard to Hackcited by 11
  11. 2023Scaling Laws for Imitation Learning in Single-Agent Gamescited by 11
  12. 2023diff History for Neural Language Agentscited by 6
4 more
  1. 2023Katakomba: Tools and Benchmarks for Data-Driven NetHackcited by 5
  2. 2024Syllabus: Portable Curricula for Reinforcement Learning Agentscited by 2
  3. 2023LuckyMera: a Modular AI Framework for Building Hybrid NetHack Agentscited by 0
  4. 2026Hierarchical Behaviour Spacescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026NitroGen: An Open Foundation Model for Generalist Gaming Agents
  2. 2025SC2Arena and StarEvolve: Benchmark and Self-Improvement Framework for LLMs in Complex Decision-Making Tasks
  3. 2024StarCraftImage: A Dataset For Prototyping Spatial Reasoning Methods For Multi-Agent Environments
  4. 2023AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning
  5. 2026Position: Profiling Game Worlds by Transition Complexity
  6. 2024Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarks
  7. 2026Kuration SDK: Addressing the Virtual2Real Gap via Data Curation
  8. 2026World Models for Policy Refinement in StarCraft II