mlsanityData to 14 Oct 2026

Procedural Game Level Design with Deep Reinforcement Learning

arXiv 2510.15120 ↗First posted 16 Oct 2025cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Procedural content generation (PCG) has become an increasingly popular technique in game development, allowing developers to generate dynamic, replayable, and scalable environments with reduced manual effort. In this study, a novel method for procedural level design using Deep Reinforcement Learning (DRL) within a Unity-based 3D environment is proposed. The system comprises two agents: a hummingbird agent, acting as a solver, and a floating island agent, responsible for generating and placing collectible objects (flowers) on the terrain in a realistic and context-aware manner. The hummingbird is trained using the Proximal Policy Optimization (PPO) algorithm from the Unity ML-Agents toolkit. It learns to navigate through the terrain efficiently, locate flowers, and collect them while adapting to the ever-changing procedural layout of the island. The island agent is also trained using the Proximal Policy Optimization (PPO) algorithm. It learns to generate flower layouts based on observed obstacle positions, the hummingbird's initial state, and performance feedback from previous episodes. The interaction between these agents leads to emergent behavior and robust generalization across various environmental configurations. The results demonstrate that the approach not only produces effective and efficient agent behavior but also opens up new opportunities for autonomous game level design driven by machine learning. This work highlights the potential of DRL in enabling intelligent agents to both generate and solve content in virtual environments, pushing the boundaries of what AI can contribute to creative game development processes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 21 references that are in the map. First posted on arXiv 16 Oct 2025.

Builds on

14 of 21 references.

References · 14

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  7. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  8. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  9. 2020First return, then explorecited by 266
  10. 2018Unity: A General Platform for Intelligent Agentscited by 236
  11. 2018An Introduction to Deep Reinforcement Learningcited by 206
  12. 2017Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learningcited by 150
2 more
  1. 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 81
  2. 2018Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Networkcited by 76

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A modular framework for automated evaluation of procedural content generation in serious games with deep reinforcement learning agents
  2. 2025Video Game Level Design as a Multi-Agent Reinforcement Learning Problem
  3. 2024PCGRL+: Scaling, Control and Generalization in Reinforcement Learning Level Generators
  4. 2025Evolutionary Optimization of Deep Learning Agents for Sparrow Mahjong
  5. 2023PCGPT: Procedural Content Generation via Transformers
  6. 2025Policy Improvement with Style-Specific Demonstrations
  7. 2022Generative Personas That Behave and Experience Like Humans
  8. 2025PPO-ACT: Proximal Policy Optimization with Adversarial Curriculum Transfer for Spatial Public Goods Games