# Patchview: LLM-Powered Worldbuilding with Generative Dust and Magnet Visualization

arXiv: https://arxiv.org/abs/2408.04112 · HTML: https://mlsanity.com/paper/2408.04112/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 62 references that are in the map.
- First posted on arXiv 7 Aug 2024.

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [Fine-Grained Human Feedback Gives Better Rewards for Language Model Training](https://mlsanity.com/paper/2306.01693/) (2023)
- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals](https://mlsanity.com/paper/2209.14958/) (2022)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [Homogenization Effects of Large Language Models on Human Creative Ideation](https://mlsanity.com/paper/2402.01536/) (2024)
- [ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing](https://mlsanity.com/paper/2309.09128/) (2023)
- [A Design Space for Intelligent and Interactive Writing Assistants](https://mlsanity.com/paper/2403.14117/) (2024)
- [TaleCrafter: Interactive Story Visualization with Multiple Characters](https://mlsanity.com/paper/2305.18247/) (2023)
- [Bridging the Gulf of Envisioning: Cognitive Design Challenges in LLM Interfaces](https://mlsanity.com/paper/2309.14459/) (2023)
- [Graphologue: Exploring Large Language Model Responses with Interactive Diagrams](https://mlsanity.com/paper/2305.11473/) (2023)
- [Sensecape: Enabling Multilevel Exploration and Sensemaking with Large Language Models](https://mlsanity.com/paper/2305.11483/) (2023)
- [PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions](https://mlsanity.com/paper/2308.05184/) (2023)
- [Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts](https://mlsanity.com/paper/2308.03921/) (2023)
- [Weaver: Foundation Models for Creative Writing](https://mlsanity.com/paper/2401.17268/) (2024)
- [CALYPSO: LLMs as Dungeon Masters' Assistants](https://mlsanity.com/paper/2308.07540/) (2023)
- [Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets](https://mlsanity.com/paper/2310.09985/) (2023)
- [WorldSmith: Iterative and Expressive Prompting for World Building with a Generative AI](https://mlsanity.com/paper/2308.13355/) (2023)
- [Authors' Values and Attitudes Towards AI-bridged Scalable Personalization of Creative Language Arts](https://mlsanity.com/paper/2403.00439/) (2024)
- [TaleStream: Supporting Story Ideation with Trope Knowledge](https://mlsanity.com/paper/2309.03790/) (2023)
- [Portrayal: Leveraging NLP and Visualization for Analyzing Fictional Characters](https://mlsanity.com/paper/2308.04056/) (2023)

## Built on by (most cited first, 23 of 23)

- [MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines](https://mlsanity.com/paper/2603.06679/) (2026)
- [Unbounded: A Generative Infinite Game of Character Life Simulation](https://mlsanity.com/paper/2410.18975/) (2024)
- [WhatELSE: Shaping Narrative Spaces at Configurable Level of Abstraction for AI-bridged Interactive Storytelling](https://mlsanity.com/paper/2502.18641/) (2025)
- [DeepVIS: Bridging Natural Language and Data Visualization Through Step-wise Reasoning](https://mlsanity.com/paper/2508.01700/) (2025)
- [Steering AI-Driven Personalization of Scientific Text for General Audiences](https://mlsanity.com/paper/2411.09969/) (2024)
- [AnimeGamer: Infinite Anime Life Simulation with Next Game State Prediction](https://mlsanity.com/paper/2504.01014/) (2025)
- [Where Do I 'Add the Egg'?: Exploring Agency and Ownership in AI Creative Co-Writing Systems](https://mlsanity.com/paper/2509.15440/) (2025)
- [Toyteller: AI-powered Visual Storytelling Through Toy-Playing with Character Symbols](https://mlsanity.com/paper/2501.13284/) (2025)
- [DiscoverLLM: From Executing Intents to Discovering Them](https://mlsanity.com/paper/2602.03429/) (2026)
- [Constella: Supporting Storywriters' Interconnected Character Creation through LLM-based Multi-Agents](https://mlsanity.com/paper/2507.05820/) (2025)
- [Narrix: Remixing Narrative Strategies from Examples for Story Writing](https://mlsanity.com/paper/2604.07643/) (2026)
- [Scaffolding Recursive Divergence and Convergence in Story Ideation](https://mlsanity.com/paper/2507.03307/) (2025)
- [Vidmento: Creating Video Stories Through Context-Aware Expansion With Generative Video](https://mlsanity.com/paper/2601.22013/) (2026)
- [Phys4D: Fine-Grained Physics-Consistent 4D Modeling from Video Diffusion](https://mlsanity.com/paper/2603.03485/) (2026)
- [Exploring Mobile Touch Interaction with Large Language Models](https://mlsanity.com/paper/2502.07629/) (2025)
- [TaleForge: Interactive Multimodal System for Personalized Story Creation](https://mlsanity.com/paper/2506.21832/) (2025)
- [Story Ribbons: Reimagining Storyline Visualizations with Large Language Models](https://mlsanity.com/paper/2508.06772/) (2025)
- [CareerPooler: AI-Powered Metaphorical Pool Simulation Improves Experience and Outcomes in Career Exploration](https://mlsanity.com/paper/2509.11461/) (2025)
- [PlayWrite: A Multimodal System for AI Supported Narrative Co-Authoring Through Play in XR](https://mlsanity.com/paper/2603.02366/) (2026)
- [Narrative Keyframing for Generative Creative Writing](https://mlsanity.com/paper/2608.10337/) (2026)
- [Elsewise: Authoring Open-ended Interactive Narrative with Possibility Space Visualization](https://mlsanity.com/paper/2601.15295/) (2025)
- [Beyond Compliance: How AI Could Help Creative Writers by Refusing Them](https://mlsanity.com/paper/2605.16272/) (2026)
- [VeriForge: Mitigating Latent Knowledge Gaps in Narrative Drafting via Mixed-Initiative Scaffolding](https://mlsanity.com/paper/2608.09698/) (2026)

## Abstract (arXiv)

Large language models (LLMs) can help writers build story worlds by generating world elements, such as factions, characters, and locations. However, making sense of many generated elements can be overwhelming. Moreover, if the user wants to precisely control aspects of generated elements that are difficult to specify verbally, prompting alone may be insufficient. We introduce Patchview, a customizable LLM-powered system that visually aids worldbuilding by allowing users to interact with story concepts and elements through the physical metaphor of magnets and dust. Elements in Patchview are visually dragged closer to concepts with high relevance, facilitating sensemaking. The user can also steer the generation with verbally elusive concepts by indicating the desired position of the element between concepts. When the user disagrees with the LLM's visualization and generation, they can correct those by repositioning the element. These corrections can be used to align the LLM's future behaviors to the user's perception. With a user study, we show that Patchview supports the sensemaking of world elements and steering of element generation, facilitating exploration during the worldbuilding process. Patchview provides insights on how customizable visual representation can help sensemake, steer, and align generative AI model behaviors with the user's intentions.