# Unboxing Diffusion Models for the Arts: Interactive Model Bending and Practice-Based Explainability

arXiv: https://arxiv.org/abs/2607.22428 · HTML: https://mlsanity.com/paper/2607.22428/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 19 of its 46 references that are in the map.
- First posted on arXiv 24 Jul 2026.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Diffusion Models already have a Semantic Latent Space](https://mlsanity.com/paper/2210.10960/) (2022)
- [StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation](https://mlsanity.com/paper/2312.12491/) (2023)
- [pyvene: A Library for Understanding and Improving PyTorch Models via Interventions](https://mlsanity.com/paper/2403.07809/) (2024)
- [NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals](https://mlsanity.com/paper/2407.14561/) (2024)
- [Active Divergence with Generative Deep Learning -- A Survey and Taxonomy](https://mlsanity.com/paper/2107.05599/) (2021)
- [Exploring XAI for the Arts: Explaining Latent Space in Generative Music](https://mlsanity.com/paper/2308.05496/) (2023)
- [XAIxArts Manifesto: Explainable AI for the Arts](https://mlsanity.com/paper/2502.21220/) (2025)
- [Towards Directive Explanations: Crafting Explainable AI Systems for Actionable Human-AI Interactions](https://mlsanity.com/paper/2401.04118/) (2023)
- [A Taxonomy of the Biases of the Images created by Generative Artificial Intelligence](https://mlsanity.com/paper/2407.01556/) (2024)
- [Explainability Paths for Sustained Artistic Practice with AI](https://mlsanity.com/paper/2407.15216/) (2024)
- [Hacking Generative Models with Differentiable Network Bending](https://mlsanity.com/paper/2310.04816/) (2023)
- [Network Bending of Diffusion Models for Audio-Visual Generation](https://mlsanity.com/paper/2406.19589/) (2024)
- [Controlling the image generation process with parametric activation functions](https://mlsanity.com/paper/2510.15778/) (2025)

## Abstract (arXiv)

Explainable AI (XAI) in creative practice can be less about technocentric explanation and more about enabling artists to inspect modify and debug models as part of making Yet largescale texttoimage diffusion systems are typically presented as opaque endtoend tools limiting this kind of material engagement We argue that even large models can function as creative materials when their internal structure is made visible and manipulable To support this we propose a handson approach to explainability centred on experimentation and intervention We instantiate this approach with a model bending and an interactive (inspection) interface integrated into ComfyUIs nodebased workflow including interactive layer selection and intervention controls Through qualitative and quantitative analysis of bending interventions in Stable Diffusion 15 we show how manipulating specific components of a diffusion pipeline produces relatively consistent families of visual effects allowing artists to build practical layerlevel intuition about how different parts of the model shape generated images