mlsanityData to 7 Oct 2026

One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Models

arXiv 2410.22366 ↗First posted 28 Oct 2024cs.LG cs.AI cs.CVcited by 30

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

For large language models (LLMs), sparse autoencoders (SAEs) have been shown to decompose intermediate representations that often are not interpretable directly into sparse sums of interpretable features, facilitating better control and subsequent analysis. However, similar analyses and approaches have been lacking for text-to-image models. We investigate the possibility of using SAEs to learn interpretable features for SDXL Turbo, a few-step text-to-image diffusion model. To this end, we train SAEs on the updates performed by transformer blocks within SDXL Turbo's denoising U-net in its 1-step setting. Interestingly, we find that they generalize to 4-step SDXL Turbo and even to the multi-step SDXL base model (i.e., a different model) without additional training. In addition, we show that their learned features are interpretable, causally influence the generation process, and reveal specialization among the blocks. We do so by creating RIEBench, a representation-based image editing benchmark, for editing images while they are generated by turning on and off individual SAE features. This allows us to track which transformer blocks' features are the most impactful depending on the edit category. Our work is the first investigation of SAEs for interpretability in text-to-image diffusion models and our results establish SAEs as a promising approach for understanding and manipulating the internal mechanisms of text-to-image models.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 28 of its 65 references that are in the map. First posted on arXiv 28 Oct 2024.

Builds on

28 of 65 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Adversarial Diffusion Distillation28 Nov
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  3. 2022Scalable Diffusion Models with Transformers19 Dec
  4. 2022Diffusion Models already have a Semantic Latent Space20 Oct
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May

Other references · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  7. 2020GLU Variants Improve Transformercited by 1,724
  8. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  10. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  11. 2024Scaling and evaluating sparse autoencoderscited by 530
  12. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
10 more
  1. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  2. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  3. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  4. 2023Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Modelscited by 148
  5. 2023Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Codecited by 145
  6. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  7. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 106
  8. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 63
  9. 2023Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Modelscited by 50
  10. 2023An Image is Worth Multiple Words: Multi-attribute Inversion for Constrained Text-to-Image Synthesiscited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 19.

30 papers.

Sort:

By area:

  1. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 75
  2. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Modelscited by 70
  3. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 67
  4. 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 58
  5. 2025Interpreting CLIP with Hierarchical Sparse Autoencoderscited by 54
  6. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  7. 2025Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Wordscited by 17
  8. 2025TIDE : Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generationcited by 16
  9. 2025SAEmnesia: Erasing Concepts in Diffusion Models with Supervised Sparse Autoencoderscited by 8
  10. 2026DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoderscited by 6
  11. 2026Interpretable Debiasing of Vision-Language Models for Social Fairnesscited by 6
  12. 2026ELROND: Exploring and decomposing intrinsic capabilities of diffusion modelscited by 2
18 more
  1. 2026TADA! Tuning Audio Diffusion Models through Activation Steeringcited by 2
  2. 2026DifFRACT: Diffusion Feature Reconstruction and Attribution for Circuit Tracingcited by 2
  3. 2025Beyond Interpretability: When, Why, and How Sparse Autoencoders Enable Label-Free Visual Steeringcited by 1
  4. 2026Meltdown: Circuits and Bifurcations in Point-Cloud-Conditioned 3D Diffusion Transformerscited by 1
  5. 2026Language Models Can Explain Visual Features via Steeringcited by 1
  6. 2026SoftSAE: Dynamic Top-K Selection for Adaptive Sparse Autoencoderscited by 1
  7. 2026Residualized Temporal Sparse Autoencoders for Interpreting Diffusion Modelscited by 1
  8. 2025SALVE: Sparse Autoencoder-Latent Vector Editing for Mechanistic Control of Neural Networkscited by 0
  9. 2026Supervised sparse auto-encoders for interpretable and compositional representationscited by 0
  10. 2026RAIGen: Rare Attribute Identification in Text-to-Image Generative Modelscited by 0
  11. 2026DreamReader: An Interpretability Toolkit for Text-to-Image Modelscited by 0
  12. 2026Can Cross-Layer Transcoders Replace Vision Transformer Activations? An Interpretable Perspective on Visioncited by 0
  13. 2026Grokking of Diffusion Models: Case Study on Modular Additioncited by 0
  14. 2026Look But Don't Touch with Sparse Autoencoders for Unlearning in Diffusion Modelscited by 0
  15. 2026Position: Preventing AI-Generated CSAM Necessitates New Approaches to AI Safetycited by 0
  16. 2026The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metriccited by 0
  17. 2026Interpreting Object-Dependent Concept Brittleness in Text-to-Image Diffusion Modelscited by 0
  18. 2026D-Scope: Decomposing and Steering Diffusion Transformers with Sparse Autoencoderscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning
  2. 2025Emergence and Evolution of Interpretable Concepts in Diffusion Models
  3. 2025Probing the Representational Power of Sparse Autoencoders in Vision Models
  4. 2026Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders
  5. 2025SAEdit: Token-level control for continuous image editing via Sparse AutoEncoder
  6. 2025Yuan-TecSwin: A text conditioned Diffusion model with Swin-transformer blocks
  7. 2023Denoising Diffusion Autoencoders are Unified Self-supervised Learners
  8. 2026RepFusion: Leveraging Multimodal Priors for Denoising in Representation Space