# Multi-Objective Instruction-Aware Representation Learning in Procedural Content Generation RL

arXiv: https://arxiv.org/abs/2508.09193 · HTML: https://mlsanity.com/paper/2508.09193/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 27 references that are in the map.
- First posted on arXiv 8 Aug 2025.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Multi-task Deep Reinforcement Learning with PopArt](https://mlsanity.com/paper/1809.04474/) (2018)
- [Discovered Policy Optimisation](https://mlsanity.com/paper/2210.05639/) (2022)
- [PCGRL: Procedural Content Generation via Reinforcement Learning](https://mlsanity.com/paper/2001.09212/) (2020)
- [Level Generation Through Large Language Models](https://mlsanity.com/paper/2302.05817/) (2023)
- [MarioGPT: Open-Ended Text2Level Generation through Large Language Models](https://mlsanity.com/paper/2302.05981/) (2023)
- [Learning Controllable Content Generators](https://mlsanity.com/paper/2105.02993/) (2021)
- [Learning Attention as Disentangler for Compositional Zero-shot Learning](https://mlsanity.com/paper/2303.15111/) (2023)
- [Learning Controllable 3D Level Generators](https://mlsanity.com/paper/2206.13623/) (2022)
- [The Five-Dollar Model: Generating Game Maps and Sprites from Sentence Embeddings](https://mlsanity.com/paper/2308.04052/) (2023)
- [Compositional Generalization for Multi-label Text Classification: A Data-Augmentation Approach](https://mlsanity.com/paper/2312.11276/) (2023)
- [PCGRL+: Scaling, Control and Generalization in Reinforcement Learning Level Generators](https://mlsanity.com/paper/2408.12525/) (2024)
- [DiMPLe -- Disentangled Multi-Modal Prompt Learning: Enhancing Out-Of-Distribution Alignment with Invariant and Spurious Feature Separation](https://mlsanity.com/paper/2506.21237/) (2025)
- [IPCGRL: Language-Instructed Reinforcement Learning for Procedural Level Generation](https://mlsanity.com/paper/2503.12358/) (2025)

## Built on by (most cited first, 3 of 3)

- [Human-Aligned Procedural Level Generation Reinforcement Learning via Text-Level-Sketch Shared Representation](https://mlsanity.com/paper/2508.09860/) (2025)
- [Multiverse: Language-Conditioned Multi-Game Level Blending via Shared Representation](https://mlsanity.com/paper/2603.26782/) (2026)
- [Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL](https://mlsanity.com/paper/2607.16204/) (2026)

## Abstract (arXiv)

Recent advancements in generative modeling emphasize the importance of natural language as a highly expressive and accessible modality for controlling content generation. However, existing instructed reinforcement learning for procedural content generation (IPCGRL) method often struggle to leverage the expressive richness of textual input, especially under complex, multi-objective instructions, leading to limited controllability. To address this problem, we propose textitMIPCGRL, a multi-objective representation learning method for instructed content generators, which incorporates sentence embeddings as conditions. MIPCGRL effectively trains a multi-objective embedding space by incorporating multi-label classification and multi-head regression networks. Experimental results show that the proposed method achieves up to a 13.8% improvement in controllability with multi-objective instructions. The ability to process complex instructions enables more expressive and flexible content generation.