# IPCGRL: Language-Instructed Reinforcement Learning for Procedural Level Generation

arXiv: https://arxiv.org/abs/2503.12358 · HTML: https://mlsanity.com/paper/2503.12358/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 27 references that are in the map.
- First posted on arXiv 16 Mar 2025.

## Placed in

- Deep reinforcement learning methods / [Interpretable and logic-guided RL](https://mlsanity.com/deeprl/interpretable-and-logic-guided-rl/)

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Language Conditioned Imitation Learning over Unstructured Data](https://mlsanity.com/paper/2005.07648/) (2020)
- [Multi-Task Reinforcement Learning with Context-based Representations](https://mlsanity.com/paper/2102.06177/) (2021)
- [Procedural Content Generation via Machine Learning (PCGML)](https://mlsanity.com/paper/1702.00539/) (2017)
- [Using Natural Language for Reward Shaping in Reinforcement Learning](https://mlsanity.com/paper/1903.02020/) (2019)
- [Discovered Policy Optimisation](https://mlsanity.com/paper/2210.05639/) (2022)
- [PCGRL: Procedural Content Generation via Reinforcement Learning](https://mlsanity.com/paper/2001.09212/) (2020)
- [Learning to Understand Goal Specifications by Modelling Reward](https://mlsanity.com/paper/1806.01946/) (2018)
- [Language Instructed Reinforcement Learning for Human-AI Coordination](https://mlsanity.com/paper/2304.07297/) (2023)
- [Learning Controllable Content Generators](https://mlsanity.com/paper/2105.02993/) (2021)
- [Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control](https://mlsanity.com/paper/2307.00117/) (2023)
- [Learning Controllable 3D Level Generators](https://mlsanity.com/paper/2206.13623/) (2022)
- [Meta-Reinforcement Learning via Language Instructions](https://mlsanity.com/paper/2209.04924/) (2022)
- [Language Grounded Multi-agent Reinforcement Learning with Human-interpretable Communication](https://mlsanity.com/paper/2409.17348/) (2024)
- [RaidEnv: Exploring New Challenges in Automated Content Balancing for Boss Raid Games](https://mlsanity.com/paper/2307.01676/) (2023)
- [PCGRL+: Scaling, Control and Generalization in Reinforcement Learning Level Generators](https://mlsanity.com/paper/2408.12525/) (2024)
- [Arachnophobia Exposure Therapy using Experience-driven Procedural Content Generation via Reinforcement Learning (EDPCGRL)](https://mlsanity.com/paper/2110.04146/) (2021)
- [ChatPCG: Large Language Model-Driven Reward Design for Procedural Content Generation](https://mlsanity.com/paper/2406.11875/) (2024)
- [G-PCGRL: Procedural Graph Data Generation via Reinforcement Learning](https://mlsanity.com/paper/2407.10483/) (2024)
- [PCGRLLM: Large Language Model-Driven Reward Design for Procedural Content Generation Reinforcement Learning](https://mlsanity.com/paper/2502.10906/) (2025)

## Built on by (most cited first, 4 of 4)

- [Multi-Objective Instruction-Aware Representation Learning in Procedural Content Generation RL](https://mlsanity.com/paper/2508.09193/) (2025)
- [Human-Aligned Procedural Level Generation Reinforcement Learning via Text-Level-Sketch Shared Representation](https://mlsanity.com/paper/2508.09860/) (2025)
- [Multiverse: Language-Conditioned Multi-Game Level Blending via Shared Representation](https://mlsanity.com/paper/2603.26782/) (2026)
- [Select-to-Act: Hierarchical Reinforcement Learning via Adaptive Language Guidance](https://mlsanity.com/paper/2606.22350/) (2026)

## Abstract (arXiv)

Recent research has highlighted the significance of natural language in enhancing the controllability of generative models. While various efforts have been made to leverage natural language for content generation, research on deep reinforcement learning (DRL) agents utilizing text-based instructions for procedural content generation remains limited. In this paper, we propose IPCGRL, an instruction-based procedural content generation method via reinforcement learning, which incorporates a sentence embedding model. IPCGRL fine-tunes task-specific embedding representations to effectively compress game-level conditions. We evaluate IPCGRL in a two-dimensional level generation task and compare its performance with a general-purpose embedding method. The results indicate that IPCGRL achieves up to a 21.4% improvement in controllability and a 17.2% improvement in generalizability for unseen instructions. Furthermore, the proposed method extends the modality of conditional input, enabling a more flexible and expressive interaction framework for procedural content generation.