# Narrative-to-Scene Generation: An LLM-Driven Pipeline for 2D Game Environments

arXiv: https://arxiv.org/abs/2509.04481 · HTML: https://mlsanity.com/paper/2509.04481/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 60 references that are in the map.
- First posted on arXiv 31 Aug 2025.

## Builds on (most cited first, 20 of 20)

- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT](https://mlsanity.com/paper/2303.04226/) (2023)
- [Visual Semantic Reasoning for Image-Text Matching](https://mlsanity.com/paper/1909.02701/) (2019)
- [Procedural Content Generation via Machine Learning (PCGML)](https://mlsanity.com/paper/1702.00539/) (2017)
- [Large Language Models and Games: A Survey and Roadmap](https://mlsanity.com/paper/2402.18659/) (2024)
- [Narrative Planning: Balancing Plot and Character](https://mlsanity.com/paper/1401.3841/) (2014)
- [Knowledge-Enriched Visual Storytelling](https://mlsanity.com/paper/1912.01496/) (2019)
- [Game Generation via Large Language Models](https://mlsanity.com/paper/2404.08706/) (2024)
- [GAVEL: Generating Games Via Evolution and Language Models](https://mlsanity.com/paper/2407.09388/) (2024)
- [Word2World: Generating Stories and Worlds through Large Language Models](https://mlsanity.com/paper/2405.06686/) (2024)
- [Large language models can segment narrative events similarly to humans](https://mlsanity.com/paper/2301.10297/) (2023)
- [Tile Embedding: A General Representation for Procedural Level Generation via Machine Learning](https://mlsanity.com/paper/2110.03181/) (2021)
- [Collaborative Comic Generation: Integrating Visual Narrative Theories with AI Models for Enhanced Creativity](https://mlsanity.com/paper/2409.17263/) (2024)
- [Hierarchical Knowledge Graphs for Story Understanding in Visual Narratives](https://mlsanity.com/paper/2506.10008/) (2025)
- [Let CONAN tell you a story: Procedural quest generation](https://mlsanity.com/paper/1808.06217/) (2018)
- [A Customizable Generator for Comic-Style Visual Narrative](https://mlsanity.com/paper/2401.02863/) (2023)
- [GameTileNet: A Semantic Dataset for Low-Resolution Game Art in Procedural Content Generation](https://mlsanity.com/paper/2507.02941/) (2025)
- [All Stories Are One Story: Emotional Arc Guided Procedural Game Level Generation](https://mlsanity.com/paper/2508.02132/) (2025)
- [Robust Symbolic Reasoning for Visual Narratives via Hierarchical and Semantically Normalized Knowledge Graphs](https://mlsanity.com/paper/2508.14941/) (2025)

## Built on by (most cited first, 2 of 2)

- [MAGIC: Transition-Aware Generation of Navigable Multi-Scene Game Worlds with Large Language Models](https://mlsanity.com/paper/2607.11594/) (2026)
- [Zero-shot 3D Map Generation with LLM Agents: A Dual-Agent Architecture for Procedural Content Generation](https://mlsanity.com/paper/2512.10501/) (2025)

## Abstract (arXiv)

Recent advances in large language models (LLMs) enable compelling story generation, but connecting narrative text to playable visual environments remains an open challenge in procedural content generation (PCG). We present a lightweight pipeline that transforms short narrative prompts into a sequence of 2D tile-based game scenes, reflecting the temporal structure of stories. Given an LLM-generated narrative, our system identifies three key time frames, extracts spatial predicates in the form of "Object-Relation-Object" triples, and retrieves visual assets using affordance-aware semantic embeddings from the GameTileNet dataset. A layered terrain is generated using Cellular Automata, and objects are placed using spatial rules grounded in the predicate structure. We evaluated our system in ten diverse stories, analyzing tile-object matching, affordance-layer alignment, and spatial constraint satisfaction across frames. This prototype offers a scalable approach to narrative-driven scene generation and lays the foundation for future work on multi-frame continuity, symbolic tracking, and multi-agent coordination in story-centered PCG.