mlsanityData to 14 Oct 2026

Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation

arXiv 1806.10729 ↗First posted 28 Jun 2018cs.LG cs.AI stat.MLcited by 81

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (RL) has shown impressive results in a variety of domains, learning directly from high-dimensional sensory streams. However, when neural networks are trained in a fixed environment, such as a single level in a video game, they will usually overfit and fail to generalize to new levels. When RL models overfit, even slight modifications to the environment can result in poor agent performance. This paper explores how procedurally generated levels during training can increase generality. We show that for some games procedural level generation enables generalization to new levels within the same distribution. Additionally, it is possible to achieve better performance with less data by manipulating the difficulty of the levels in response to the performance of the agent. The generality of the learned behaviors is also evaluated on a set of human-designed levels. The results suggest that the ability to generalize to human-designed levels highly depends on the design of the level generators. We apply dimensionality reduction and clustering techniques to visualize the generators' distributions of levels and analyze to what degree they can produce levels similar to those designed by a human.

Key facts. Cited by 81 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 24; 2022: 10; 2023: 14; 2024: 14; 2025: 16; 2026: 2.

81 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  2. 2021Domain Generalization with MixStylecited by 566
  3. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  4. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
  5. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  6. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
  7. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 109
  8. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 93
  9. 2024Video as the New Language for Real-World Decision Makingcited by 87
  10. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 82
  11. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 80
  12. 2021MixStyle Neural Networks for Domain Generalization and Adaptationcited by 64
38 more
  1. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 56
  2. 2021Adversarially Guided Actor-Criticcited by 47
  3. 2025DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM Post-trainingcited by 46
  4. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 43
  5. 2021Learning Controllable Content Generatorscited by 40
  6. 2022Deep Surrogate Assisted Generation of Environmentscited by 40
  7. 2021Adversarial Reinforcement Learning for Procedural Content Generationcited by 36
  8. 2021Procedural Generalization by Planning with Self-Supervised World Modelscited by 31
  9. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 31
  10. 2023The Generalization Gap in Offline Reinforcement Learningcited by 26
  11. 2021Quantifying the effects of environment and population diversity in multi-agent reinforcement learningcited by 22
  12. 2022Benchmarking Reinforcement Learning Techniques for Autonomous Navigationcited by 19
  13. 2021TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RLcited by 18
  14. 2022Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worldscited by 17
  15. 2023Multi-Robot Coordination and Layout Design for Automated Warehousingcited by 17
  16. 2024DreamCraft: Text-Guided Generation of Functional 3D Environments in Minecraftcited by 17
  17. 2025Horizon Generalization in Reinforcement Learningcited by 17
  18. 2022It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generationcited by 14
  19. 2025Automated Skill Discovery for Language Agents through Exploration and Iterative Feedbackcited by 12
  20. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11
  21. 2021Situated Dialogue Learning through Procedural Environment Generationcited by 9
  22. 2022Learning Robust Real-Time Cultural Transmission without Human Datacited by 9
  23. 2022Local Feature Swapping for Generalization in Reinforcement Learningcited by 9
  24. 2023Arbitrarily Scalable Environment Generators via Neural Cellular Automatacited by 9
  25. 2023Diverse Multimedia Layout Generation with Multi Choice Learningcited by 8
  26. 2021Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learningcited by 7
  27. 2024Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PCcited by 7
  28. 2020Demonstration-efficient Inverse Reinforcement Learning in Procedurally Generated Environmentscited by 6
  29. 2021Level Generation for Angry Birds with Sequential VAE and Latent Variable Evolutioncited by 6
  30. 2021Scenic4RL: Programmatic Modeling and Generation of Reinforcement Learning Environmentscited by 6
  31. 2022Transfer Dynamics in Emergent Evolutionary Curriculacited by 6
  32. 2025Causally Aligned Curriculum Learningcited by 6
  33. 2023Learning Coverage Paths in Unknown Environments with Deep Reinforcement Learningcited by 5
  34. 2024Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalizationcited by 5
  35. 2021Policy Fusion for Adaptive and Customizable Reinforcement Learning Agentscited by 4
  36. 2023Hierarchically Composing Level Generators for the Creation of Complex Structurescited by 4
  37. 2023Improving Generalization in Game Agents with Data Augmentation in Imitation Learningcited by 4
  38. 2025PuzzleJAX: A Benchmark for Reasoning and Learningcited by 4

50 most cited of 81 All 81 (JSON)