mlsanityData to 14 Oct 2026

Video Game Level Design as a Multi-Agent Reinforcement Learning Problem

arXiv 2510.04862 ↗First posted 6 Oct 2025cs.AI cs.LG cs.MAcited by 1

Not on the map. Why?

Abstract (arXiv)

Procedural Content Generation via Reinforcement Learning (PCGRL) offers a method for training controllable level designer agents without the need for human datasets, using metrics that serve as proxies for level quality as rewards. Existing PCGRL research focuses on single generator agents, but are bottlenecked by the need to frequently recalculate heuristics of level quality and the agent's need to navigate around potentially large maps. By framing level generation as a multi-agent problem, we mitigate the efficiency bottleneck of single-agent PCGRL by reducing the number of reward calculations relative to the number of agent actions. We also find that multi-agent level generators are better able to generalize to out-of-distribution map shapes, which we argue is due to the generators' learning more local, modular design policies. We conclude that treating content generation as a distributed, multi-agent task is beneficial for generating functional artifacts at scale.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 6 Oct 2025.

Builds on

24 of 34 references.

References · 24

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  2. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 311
  3. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  4. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  5. 2017Procedural Content Generation via Machine Learning (PCGML)cited by 137
  6. 2020PCGRL: Procedural Content Generation via Reinforcement Learningcited by 90
  7. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 63
  8. 2017Guided Deep Reinforcement Learning for Swarm Systemscited by 44
  9. 2016Super Mario as a String: Platformer Level Generation Via LSTMscited by 40
  10. 2021Learning Controllable Content Generatorscited by 40
  11. 2019Bootstrapping Conditional GANs for Video Game Level Generationcited by 35
  12. 2019Friend, Collaborator, Student, Manager: How Design of an AI-Driven Game Level Editor Affects Creatorscited by 31
12 more
  1. 2021Generating and Blending Game Levels via Quality-Diversity in the Latent Space of a Variational Autoencodercited by 27
  2. 2020Interactive Evolution and Exploration Within Latent Level-Design Space of Generative Adversarial Networkscited by 22
  3. 2020Baba is Y'all: Collaborative Mixed-Initiative Level Designcited by 20
  4. 2018Co-Creative Level Design via Machine Learningcited by 18
  5. 2023The Five-Dollar Model: Generating Game Maps and Sprites from Sentence Embeddingscited by 14
  6. 2020Mixed-Initiative Level Design with RL Brushcited by 13
  7. 2024JaxUED: A simple and useable UED library in Jaxcited by 11
  8. 2024DreamGarden: A Designer Assistant for Growing Games from a Single Promptcited by 10
  9. 2024PCGRL+: Scaling, Control and Generalization in Reinforcement Learning Level Generatorscited by 7
  10. 2024Making New Connections: LLMs as Puzzle Generators for The New York Times' Connections Word Gamecited by 4
  11. 2020Machine learning based co-creative design frameworkcited by 1
  12. 2025Procedural city modelingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026RL-Lock: Reinforcement Learning for Generating Interlocking Assembliescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023PCGPT: Procedural Content Generation via Transformers
  2. 2025Zero-shot 3D Map Generation with LLM Agents: A Dual-Agent Architecture for Procedural Content Generation
  3. 2025A modular framework for automated evaluation of procedural content generation in serious games with deep reinforcement learning agents
  4. 2024G-PCGRL: Procedural Graph Data Generation via Reinforcement Learning
  5. 2024Moonshine: Distilling Game Content Generators into Steerable Generative Models
  6. 2025PCGRLLM: Large Language Model-Driven Reward Design for Procedural Content Generation Reinforcement Learning
  7. 2025IPCGRL: Language-Instructed Reinforcement Learning for Procedural Level Generation
  8. 2026GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?