# Text-to-Level Diffusion Models With Various Text Encoders for Super Mario Bros

arXiv: https://arxiv.org/abs/2507.00184 · HTML: https://mlsanity.com/paper/2507.00184/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 30 Jun 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 17 of 17)

- [Diffusion Models Are Real-Time Game Engines](https://mlsanity.com/paper/2408.14837/) (2024)
- [Procedural Content Generation via Machine Learning (PCGML)](https://mlsanity.com/paper/1702.00539/) (2017)
- [Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Network](https://mlsanity.com/paper/1805.00728/) (2018)
- [Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems](https://mlsanity.com/paper/2012.03448/) (2020)
- [The VGLC: The Video Game Level Corpus](https://mlsanity.com/paper/1606.07487/) (2016)
- [Super Mario as a String: Platformer Level Generation Via LSTMs](https://mlsanity.com/paper/1603.00930/) (2016)
- [On the Importance of Environments in Human-Robot Coordination](https://mlsanity.com/paper/2106.10853/) (2021)
- [Interactive Evolution and Exploration Within Latent Level-Design Space of Generative Adversarial Networks](https://mlsanity.com/paper/2004.00151/) (2020)
- [CPPN2GAN: Combining Compositional Pattern Producing Networks and GANs for Large-scale Pattern Generation](https://mlsanity.com/paper/2004.01703/) (2020)
- [Co-Creative Level Design via Machine Learning](https://mlsanity.com/paper/1809.09420/) (2018)
- [Conditional Level Generation and Game Blending](https://mlsanity.com/paper/2010.07735/) (2020)
- [Deep Interactive Evolution](https://mlsanity.com/paper/1801.08230/) (2018)
- [World-GAN: a Generative Model for Minecraft Worlds](https://mlsanity.com/paper/2106.10155/) (2021)
- [The Five-Dollar Model: Generating Game Maps and Sprites from Sentence Embeddings](https://mlsanity.com/paper/2308.04052/) (2023)
- [Generative Adversarial Network Rooms in Generative Graph Grammar Dungeons for The Legend of Zelda](https://mlsanity.com/paper/2001.05065/) (2020)
- [Hybrid Encoding For Generating Large Scale Game Level Patterns With Local Variations](https://mlsanity.com/paper/2105.12960/) (2021)
- [Moonshine: Distilling Game Content Generators into Steerable Generative Models](https://mlsanity.com/paper/2408.09594/) (2024)

## Abstract (arXiv)

Recent research shows how diffusion models can unconditionally generate tile-based game levels, but use of diffusion models for text-to-level generation is underexplored. There are practical considerations for creating a usable model: caption/level pairs are needed, as is a text embedding model, and a way of generating entire playable levels, rather than individual scenes. We present strategies to automatically assign descriptive captions to an existing dataset, and train diffusion models using both pretrained text encoders and simple transformer models trained from scratch. Captions are automatically assigned to generated scenes so that the degree of overlap between input and output captions can be compared. We also assess the diversity and playability of the resulting level scenes. Results are compared with an unconditional diffusion model and a generative adversarial network, as well as the text-to-level approaches Five-Dollar Model and MarioGPT. Notably, the best diffusion model uses a simple transformer model for text embedding, and takes less time to train than diffusion models employing more complex text encoders, indicating that reliance on larger language models is not necessary. We also present a GUI allowing designers to construct long levels from model-generated scenes.