# Insights From the NeurIPS 2021 NetHack Challenge

arXiv: https://arxiv.org/abs/2203.11889 · HTML: https://mlsanity.com/paper/2203.11889/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 16 references that are in the map.
- First posted on arXiv 22 Mar 2022.

## Builds on (most cited first, 7 of 7)

- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [Stabilizing Transformers for Reinforcement Learning](https://mlsanity.com/paper/1910.06764/) (2019)
- [The NetHack Learning Environment](https://mlsanity.com/paper/2006.13760/) (2020)
- [MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research](https://mlsanity.com/paper/2109.13202/) (2021)
- [Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning](https://mlsanity.com/paper/2006.11751/) (2020)
- [TorchBeast: A PyTorch Platform for Distributed RL](https://mlsanity.com/paper/1910.03552/) (2019)

## Built on by (most cited first, 20 of 20)

- [BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games](https://mlsanity.com/paper/2411.13543/) (2024)
- [Motif: Intrinsic Motivation from Artificial Intelligence Feedback](https://mlsanity.com/paper/2310.00166/) (2023)
- [Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning](https://mlsanity.com/paper/2402.16801/) (2024)
- [Deep Surrogate Assisted Generation of Environments](https://mlsanity.com/paper/2206.04199/) (2022)
- [RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code](https://mlsanity.com/paper/2503.07832/) (2025)
- [MaestroMotif: Skill Design from Artificial Intelligence Feedback](https://mlsanity.com/paper/2412.08542/) (2024)
- [Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problem](https://mlsanity.com/paper/2402.02868/) (2024)
- [Playing NetHack with LLMs: Potential & Limitations as Zero-Shot Agents](https://mlsanity.com/paper/2403.00690/) (2024)
- [Dungeons and Data: A Large-Scale NetHack Dataset](https://mlsanity.com/paper/2211.00539/) (2022)
- [NetHack is Hard to Hack](https://mlsanity.com/paper/2305.19240/) (2023)
- [Scaling Laws for Imitation Learning in Single-Agent Games](https://mlsanity.com/paper/2307.09423/) (2023)
- [Accelerating exploration and representation learning with offline pre-training](https://mlsanity.com/paper/2304.00046/) (2023)
- [diff History for Neural Language Agents](https://mlsanity.com/paper/2312.07540/) (2023)
- [Katakomba: Tools and Benchmarks for Data-Driven NetHack](https://mlsanity.com/paper/2306.08772/) (2023)
- [GriddlyJS: A Web IDE for Reinforcement Learning](https://mlsanity.com/paper/2207.06105/) (2022)
- [Syllabus: Portable Curricula for Reinforcement Learning Agents](https://mlsanity.com/paper/2411.11318/) (2024)
- [Probing for Consciousness in Machines](https://mlsanity.com/paper/2411.16262/) (2024)
- [LuckyMera: a Modular AI Framework for Building Hybrid NetHack Agents](https://mlsanity.com/paper/2307.08532/) (2023)
- [Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMO](https://mlsanity.com/paper/2308.15802/) (2023)
- [Hierarchical Behaviour Spaces](https://mlsanity.com/paper/2604.24558/) (2026)

## Abstract (arXiv)

In this report, we summarize the takeaways from the first NeurIPS 2021 NetHack Challenge. Participants were tasked with developing a program or agent that can win (i.e., 'ascend' in) the popular dungeon-crawler game of NetHack by interacting with the NetHack Learning Environment (NLE), a scalable, procedurally generated, and challenging Gym environment for reinforcement learning (RL). The challenge showcased community-driven progress in AI with many diverse approaches significantly beating the previously best results on NetHack. Furthermore, it served as a direct comparison between neural (e.g., deep RL) and symbolic AI, as well as hybrid systems, demonstrating that on NetHack symbolic bots currently outperform deep RL by a large margin. Lastly, no agent got close to winning the game, illustrating NetHack's suitability as a long-term benchmark for AI research.