# Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts

arXiv: https://arxiv.org/abs/2308.03921 · HTML: https://mlsanity.com/paper/2308.03921/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 81 references that are in the map.
- First posted on arXiv 7 Aug 2023.

## Builds on (most cited first, 14 of 14)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://mlsanity.com/paper/2109.06977/) (2021)
- [Grounded Copilot: How Programmers Interact with Code-Generating Models](https://mlsanity.com/paper/2206.15000/) (2022)
- [Foundation Models and Fair Use](https://mlsanity.com/paper/2303.15715/) (2023)
- [Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming](https://mlsanity.com/paper/2210.14306/) (2022)
- [Sketch2Code: Transformation of Sketches to UI in Real-time Using Deep Neural Network](https://mlsanity.com/paper/1910.08930/) (2019)
- [The Prompt Artists](https://mlsanity.com/paper/2303.12253/) (2023)
- [A Word is Worth a Thousand Pictures: Prompts as AI Design Material](https://mlsanity.com/paper/2303.12647/) (2023)
- [What We Can Learn From Visual Artists About Software Development](https://mlsanity.com/paper/2101.11218/) (2021)

## Built on by (most cited first, 30 of 30)

- [OpenAgents: An Open Platform for Language Agents in the Wild](https://mlsanity.com/paper/2310.10634/) (2023)
- [WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualization](https://mlsanity.com/paper/2408.01703/) (2024)
- [Cocoa: Co-Planning and Co-Execution with AI Agents](https://mlsanity.com/paper/2412.10999/) (2024)
- [Patchview: LLM-Powered Worldbuilding with Generative Dust and Magnet Visualization](https://mlsanity.com/paper/2408.04112/) (2024)
- [Natural Language Outlines for Code: Literate Programming in the LLM Era](https://mlsanity.com/paper/2408.04820/) (2024)
- [JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models](https://mlsanity.com/paper/2404.08793/) (2024)
- [COMFYCLAW: Self-Evolving Skill Harnesses for Image Generation Workflows](https://mlsanity.com/paper/2607.01709/) (2026)
- [Human-AI Experience in Integrated Development Environments: A Systematic Literature Review](https://mlsanity.com/paper/2503.06195/) (2025)
- [LLM Code Customization with Visual Results: A Benchmark on TikZ](https://mlsanity.com/paper/2505.04670/) (2025)
- [AI-Assisted Causal Pathway Diagram for Human-Centered Design](https://mlsanity.com/paper/2403.08111/) (2024)
- [Large Language Models Meet User Interfaces: The Case of Provisioning Feedback](https://mlsanity.com/paper/2404.11072/) (2024)
- [A Systematic Review of Human-AI Co-Creativity](https://mlsanity.com/paper/2506.21333/) (2025)
- [Pinning "Reflection" on the Agenda: Investigating Reflection in Human-LLM Co-Creation for Creative Coding](https://mlsanity.com/paper/2402.09750/) (2024)
- [Benchmarking Generative Models on Computational Thinking Tests in Elementary Visual Programming](https://mlsanity.com/paper/2406.09891/) (2024)
- [Composable Prompting Workspaces for Creative Writing: Exploration and Iteration Using Dynamic Widgets](https://mlsanity.com/paper/2503.21394/) (2025)
- [MapStory: Prototyping Editable Map Animations with LLM Agents](https://mlsanity.com/paper/2505.21966/) (2025)
- [FeedQUAC: Quick Unobtrusive AI-Generated Commentary](https://mlsanity.com/paper/2504.16416/) (2025)
- [PromptCanvas: Composable Prompting Workspaces Using Dynamic Widgets for Exploration and Iteration in Creative Writing](https://mlsanity.com/paper/2506.03741/) (2025)
- [Interactive Program Synthesis for Modeling Collaborative Physical Activities from Narrated Demonstrations](https://mlsanity.com/paper/2509.24250/) (2025)
- [PortfolioMentor: Multimodal Generative AI Companion for Learning and Crafting Interactive Digital Art Portfolios](https://mlsanity.com/paper/2311.14091/) (2023)
- [Large Language Models as Partners in Student Essay Evaluation](https://mlsanity.com/paper/2405.18632/) (2024)
- [SHAPE-IT: Exploring Text-to-Shape-Display for Generative Shape-Changing Behaviors with LLMs](https://mlsanity.com/paper/2409.06205/) (2024)
- [AgentDynEx: Nudging the Mechanics and Dynamics of Multi-Agent Simulations](https://mlsanity.com/paper/2504.09662/) (2025)
- [Orchid: Orchestrating Context Across Creative Workflows with Generative AI](https://mlsanity.com/paper/2508.19517/) (2025)
- [From State Changes to Creative Decisions: Documenting and Interpreting Traces Across Creative Domains](https://mlsanity.com/paper/2603.07184/) (2026)
- [GenFlow: Interactive Modular System for Image Generation](https://mlsanity.com/paper/2506.21369/) (2025)
- [ChatGraPhT: A Visual Conversation Interface for Multi-Path Reflection with Agentic LLM Support](https://mlsanity.com/paper/2512.22790/) (2025)
- [Compos3D: Interactive Part-Based Composition for Creative Control in Generative 3D Models](https://mlsanity.com/paper/2607.12193/) (2026)
- [AdaLens: Interactive Storyline for Monitoring and Steering Long-Running Agentic Data Analysis](https://mlsanity.com/paper/2608.17834/) (2026)
- [Method, Mind, and Morality: How People Make Sense of Artificial Intelligence](https://mlsanity.com/paper/2608.24748/) (2026)

## Abstract (arXiv)

Creative coding tasks are often exploratory in nature. When producing digital artwork, artists usually begin with a high-level semantic construct such as a "stained glass filter" and programmatically implement it by varying code parameters such as shape, color, lines, and opacity to produce visually appealing results. Based on interviews with artists, it can be effortful to translate semantic constructs to program syntax, and current programming tools don't lend well to rapid creative exploration. To address these challenges, we introduce Spellburst, a large language model (LLM) powered creative-coding environment. Spellburst provides (1) a node-based interface that allows artists to create generative art and explore variations through branching and merging operations, (2) expressive prompt-based interactions to engage in semantic programming, and (3) dynamic prompt-driven interfaces and direct code editing to seamlessly switch between semantic and syntactic exploration. Our evaluation with artists demonstrates Spellburst's potential to enhance creative coding practices and inform the design of computational creativity tools that bridge semantic and syntactic spaces.