# The Five-Dollar Model: Generating Game Maps and Sprites from Sentence Embeddings

arXiv: https://arxiv.org/abs/2308.04052 · HTML: https://mlsanity.com/paper/2308.04052/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 23 references that are in the map.
- First posted on arXiv 8 Aug 2023.

## Builds on (most cited first, 21 of 21)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.14704/) (2022)
- [Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations](https://mlsanity.com/paper/1807.01697/) (2018)
- [D2C: Diffusion-Denoising Models for Few-shot Conditional Generation](https://mlsanity.com/paper/2106.06819/) (2021)
- [Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Network](https://mlsanity.com/paper/1805.00728/) (2018)
- [Level Generation Through Large Language Models](https://mlsanity.com/paper/2302.05817/) (2023)
- [Illuminating Diverse Neural Cellular Automata for Level Generation](https://mlsanity.com/paper/2109.05489/) (2021)
- [Generating and Blending Game Levels via Quality-Diversity in the Latent Space of a Variational Autoencoder](https://mlsanity.com/paper/2102.12463/) (2021)
- [Pixel VQ-VAEs for Improved Pixel Art Representation](https://mlsanity.com/paper/2203.12130/) (2022)
- [Aesthetic Bot: Interactively Evolving Game Maps on Twitter](https://mlsanity.com/paper/2208.05017/) (2022)

## Built on by (most cited first, 14 of 14)

- [Procedural Content Generation in Games: A Survey with Insights on Emerging LLM Integration](https://mlsanity.com/paper/2410.15644/) (2024)
- [DreamCraft: Text-Guided Generation of Functional 3D Environments in Minecraft](https://mlsanity.com/paper/2404.15538/) (2024)
- [The Ink Splotch Effect: A Case Study on ChatGPT as a Co-Creative Game Designer](https://mlsanity.com/paper/2403.02454/) (2024)
- [GPT for Games: A Scoping Review (2020-2023)](https://mlsanity.com/paper/2404.17794/) (2024)
- [DreamGarden: A Designer Assistant for Growing Games from a Single Prompt](https://mlsanity.com/paper/2410.01791/) (2024)
- [GPT for Games: An Updated Scoping Review (2020-2024)](https://mlsanity.com/paper/2411.00308/) (2024)
- [From Words to Worlds: Transforming One-line Prompt into Immersive Multi-modal Digital Stories with Communicative LLM Agent](https://mlsanity.com/paper/2406.10478/) (2024)
- [Moonshine: Distilling Game Content Generators into Steerable Generative Models](https://mlsanity.com/paper/2408.09594/) (2024)
- [Multi-Objective Instruction-Aware Representation Learning in Procedural Content Generation RL](https://mlsanity.com/paper/2508.09193/) (2025)
- [Higher Replay Ratio Empowers Sample-Efficient Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2404.09715/) (2024)
- [Video Game Level Design as a Multi-Agent Reinforcement Learning Problem](https://mlsanity.com/paper/2510.04862/) (2025)
- [Multiverse: Language-Conditioned Multi-Game Level Blending via Shared Representation](https://mlsanity.com/paper/2603.26782/) (2026)
- [In the Blink of an Eye: Instant Game Map Editing using a Generative-AI Smart Brush](https://mlsanity.com/paper/2503.19793/) (2025)
- [Text-to-Level Diffusion Models With Various Text Encoders for Super Mario Bros](https://mlsanity.com/paper/2507.00184/) (2025)

## Abstract (arXiv)

The five-dollar model is a lightweight text-to-image generative architecture that generates low dimensional images from an encoded text prompt. This model can successfully generate accurate and aesthetically pleasing content in low dimensional domains, with limited amounts of training data. Despite the small size of both the model and datasets, the generated images are still able to maintain the encoded semantic meaning of the textual prompt. We apply this model to three small datasets: pixel art video game maps, video game sprite images, and down-scaled emoji images and apply novel augmentation strategies to improve the performance of our model on these limited datasets. We evaluate our models performance using cosine similarity score between text-image pairs generated by the CLIP VIT-B/32 model.