# Rogue-Gym: A New Challenge for Generalization in Reinforcement Learning

arXiv: https://arxiv.org/abs/1904.08129 · HTML: https://mlsanity.com/paper/1904.08129/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2019.

## Built on by (most cited first, 6 of 6)

- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research](https://mlsanity.com/paper/2109.13202/) (2021)
- [Divide-Fuse-Conquer: Eliciting "Aha Moments" in Multi-Scenario Games](https://mlsanity.com/paper/2505.16401/) (2025)
- [Sparse Attention Guided Dynamic Value Estimation for Single-Task Multi-Scene Reinforcement Learning](https://mlsanity.com/paper/2102.07266/) (2021)
- [Training RL Agents for Multi-Objective Network Defense Tasks](https://mlsanity.com/paper/2505.22531/) (2025)
- [Mode-Dependent Rectification for Stable PPO Training](https://mlsanity.com/paper/2602.05619/) (2026)

## Abstract (arXiv)

In this paper, we propose Rogue-Gym, a simple and classic style roguelike game built for evaluating generalization in reinforcement learning (RL). Combined with the recent progress of deep neural networks, RL has successfully trained human-level agents without human knowledge in many games such as those for Atari 2600. However, it has been pointed out that agents trained with RL methods often overfit the training environment, and they work poorly in slightly different environments. To investigate this problem, some research environments with procedural content generation have been proposed. Following these studies, we propose the use of roguelikes as a benchmark for evaluating the generalization ability of RL agents. In our Rogue-Gym, agents need to explore dungeons that are structured differently each time they start a new game. Thanks to the very diverse structures of the dungeons, we believe that the generalization benchmark of Rogue-Gym is sufficiently fair. In our experiments, we evaluate a standard reinforcement learning method, PPO, with and without enhancements for generalization. The results show that some enhancements believed to be effective fail to mitigate the overfitting in Rogue-Gym, although others slightly improve the generalization ability.