mlsanityData to 7 Oct 2026

Katakomba: Tools and Benchmarks for Data-Driven NetHack

arXiv 2306.08772 ↗First posted 14 Jun 2023cs.LG cs.AI cs.NEcited by 5

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

NetHack is known as the frontier of reinforcement learning research where learning-based methods still need to catch up to rule-based solutions. One of the promising directions for a breakthrough is using pre-collected datasets similar to recent developments in robotics, recommender systems, and more under the umbrella of offline reinforcement learning (ORL). Recently, a large-scale NetHack dataset was released; while it was a necessary step forward, it has yet to gain wide adoption in the ORL community. In this work, we argue that there are three major obstacles for adoption: resource-wise, implementation-wise, and benchmark-wise. To address them, we develop an open-source library that provides workflow fundamentals familiar to the ORL community: pre-defined D4RL-style tasks, uncluttered baseline implementations, and reliable evaluation tools with accompanying configs and logs synced to the cloud.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 32 of its 49 references that are in the map. First posted on arXiv 14 Jun 2023.

Builds on

32 of 49 references.

Closest ancestors · 16

Same direction, newest first

  1. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learning16 May
  2. 2023Anti-Exploration by Random Network Distillation31 Jan
  3. 2022Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizes28 Nov
  4. 2022CORL: Research-oriented Deep Offline Reinforcement Learning Library13 Oct
  5. 2022Offline RL Policies Should be Trained to be Adaptive5 Jul
  6. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations9 Jun
  7. 2022Latent-Variable Advantage-Weighted Policy Optimization for Offline RL16 Mar
  8. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  9. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble4 Oct
  10. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learning22 Sep
  11. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  12. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning1 Feb
4 more
  1. 2020PLAS: Latent Action Space for Offline Reinforcement Learning14 Nov
  2. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  4. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 16

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  3. 2023Mastering Diverse Domains through World Modelscited by 1,193
  4. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  5. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  6. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 397
  7. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  8. 2020Reinforcement Learning for Combinatorial Optimization: A Surveycited by 235
  9. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
  10. 2020The NetHack Learning Environmentcited by 196
  11. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
  12. 2019Deep Reinforcement Learning for Smart Home Energy Managementcited by 28
4 more
  1. 2022Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flowscited by 21
  2. 2022Insights From the NeurIPS 2021 NetHack Challengecited by 20
  3. 2022Dungeons and Data: A Large-Scale NetHack Datasetcited by 16
  4. 2023NetHack is Hard to Hackcited by 11

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 54
  2. 2023Emergence of In-Context Reinforcement Learning from Noise Distillationcited by 26
  3. 2024XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learningcited by 13
  4. 2025Yes, Q-learning Helps Offline In-Context RLcited by 8
  5. 2024Distilling LLMs' Decomposition Abilities into Compact Language Modelscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2023Improving and Benchmarking Offline Reinforcement Learning Algorithms
  2. 2024D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning
  3. 2025NeoRL-2: Near Real-World Benchmarks for Offline Reinforcement Learning with Extended Realistic Scenarios
  4. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learning
  5. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
  6. 2024Simple Ingredients for Offline Reinforcement Learning
  7. 2023Small Dataset, Big Gains: Enhancing Reinforcement Learning by Offline Pre-Training with Model Based Augmentation
  8. 2022Leveraging Offline Data in Online Reinforcement Learning