Unboxing Diffusion Models for the Arts: Interactive Model Bending and Practice-Based Explainability
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Explainable AI (XAI) in creative practice can be less about technocentric explanation and more about enabling artists to inspect modify and debug models as part of making Yet largescale texttoimage diffusion systems are typically presented as opaque endtoend tools limiting this kind of material engagement We argue that even large models can function as creative materials when their internal structure is made visible and manipulable To support this we propose a handson approach to explainability centred on experimentation and intervention We instantiate this approach with a model bending and an interactive (inspection) interface integrated into ComfyUIs nodebased workflow including interactive layer selection and intervention controls Through qualitative and quantitative analysis of bending interventions in Stable Diffusion 15 we show how manipulating specific components of a diffusion pipeline produces relatively consistent families of visual effects allowing artists to build practical layerlevel intuition about how different parts of the model shape generated images
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 19 of its 46 references that are in the map. First posted on arXiv 24 Jul 2026.
Builds on
19 of 46 references.
References · 19
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Diffusion Models already have a Semantic Latent Spacecited by 328
- 2023StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generationcited by 58
- 2024pyvene: A Library for Understanding and Improving PyTorch Models via Interventionscited by 54
- 2024NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internalscited by 45
- 2021Active Divergence with Generative Deep Learning -- A Survey and Taxonomycited by 12
- 2023Exploring XAI for the Arts: Explaining Latent Space in Generative Musiccited by 8
7 more
- 2025XAIxArts Manifesto: Explainable AI for the Artscited by 7
- 2023Towards Directive Explanations: Crafting Explainable AI Systems for Actionable Human-AI Interactionscited by 6
- 2024A Taxonomy of the Biases of the Images created by Generative Artificial Intelligencecited by 4
- 2024Explainability Paths for Sustained Artistic Practice with AIcited by 4
- 2023Hacking Generative Models with Differentiable Network Bendingcited by 3
- 2024Network Bending of Diffusion Models for Audio-Visual Generationcited by 3
- 2025Controlling the image generation process with parametric activation functionscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Explainability-in-Action: Enabling Expressive Manipulation and Tacit Understanding by Bending Diffusion Models in ComfyUI
- 2025The Cow of Rembrandt - Analyzing Artistic Prompt Interpretation in Text-to-Image Models
- 2024Interactive Visual Learning for Stable Diffusion
- 2024KnobGen: Controlling the Sophistication of Artwork in Sketch-Based Diffusion Models
- 2025Blending Concepts with Text-to-Image Diffusion Models
- 2026AttentionBender: Manipulating Cross-Attention in Video Diffusion Transformers as a Creative Probe
- 2023Diffusion Explainer: Visual Explanation for Text-to-image Stable Diffusion
- 2023Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models