# On Mechanistic Knowledge Localization in Text-to-Image Generative Models

arXiv: https://arxiv.org/abs/2405.01008 · HTML: https://mlsanity.com/paper/2405.01008/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 18 references that are in the map.
- First posted on arXiv 2 May 2024.

## Builds on (most cited first, 17 of 17)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Erasing Concepts from Diffusion Models](https://mlsanity.com/paper/2303.07345/) (2023)
- [Ablating Concepts in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.13516/) (2023)
- [Direct and Indirect Effects](https://mlsanity.com/paper/1301.2300/) (2013)
- [What the DAAM: Interpreting Stable Diffusion Using Cross Attention](https://mlsanity.com/paper/2210.04885/) (2022)
- [Editing Implicit Assumptions in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.08084/) (2023)
- [Localizing and Editing Knowledge in Text-to-Image Generative Models](https://mlsanity.com/paper/2310.13730/) (2023)
- [The Hidden Language of Diffusion Models](https://mlsanity.com/paper/2306.00966/) (2023)

## Built on by (most cited first, 28 of 28)

- [SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders](https://mlsanity.com/paper/2501.18052/) (2025)
- [Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space](https://mlsanity.com/paper/2406.19370/) (2024)
- [A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models](https://mlsanity.com/paper/2502.17516/) (2025)
- [textitRevelio: Interpreting and leveraging semantic information in diffusion models](https://mlsanity.com/paper/2411.16725/) (2024)
- [TIDE : Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generation](https://mlsanity.com/paper/2503.07050/) (2025)
- [Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrations](https://mlsanity.com/paper/2511.22697/) (2025)
- [Unveiling Concept Attribution in Diffusion Models](https://mlsanity.com/paper/2412.02542/) (2024)
- [Graph-based Unsupervised Disentangled Representation Learning via Multimodal Large Language Models](https://mlsanity.com/paper/2407.18999/) (2024)
- [SAEmnesia: Erasing Concepts in Diffusion Models with Supervised Sparse Autoencoders](https://mlsanity.com/paper/2509.21379/) (2025)
- [Safe and Reliable Diffusion Models via Subspace Projection](https://mlsanity.com/paper/2503.16835/) (2025)
- [Localizing Knowledge in Diffusion Transformers](https://mlsanity.com/paper/2505.18832/) (2025)
- [Precise Parameter Localization for Textual Generation in Diffusion Models](https://mlsanity.com/paper/2502.09935/) (2025)
- [Survey of User Interface Design and Interaction Techniques in Generative AI Applications](https://mlsanity.com/paper/2410.22370/) (2024)
- [Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocols](https://mlsanity.com/paper/2507.19894/) (2025)
- [TADA! Tuning Audio Diffusion Models through Activation Steering](https://mlsanity.com/paper/2602.11910/) (2026)
- [Low-Rank Continual Personalization of Diffusion Models](https://mlsanity.com/paper/2410.04891/) (2024)
- [Localized Concept Erasure in Text-to-Image Diffusion Models via High-Level Representation Misdirection](https://mlsanity.com/paper/2602.19631/) (2026)
- [Selective Fine-Tuning for Targeted and Robust Concept Unlearning](https://mlsanity.com/paper/2602.07919/) (2026)
- [Attention, May I Have Your Decision? Localizing Generative Choices in Diffusion Models](https://mlsanity.com/paper/2604.06052/) (2026)
- [Rethinking Robust Adversarial Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2510.27285/) (2025)
- [Interpretable Motion-Attentive Maps: Spatio-Temporally Localizing Concepts in Video Diffusion Transformers](https://mlsanity.com/paper/2603.02919/) (2026)
- [AccessShare: Co-designing Data Access and Sharing with Blind People](https://mlsanity.com/paper/2407.19351/) (2024)
- [CA-LoRA: Concept-Aware LoRA for Domain-Aligned Segmentation Dataset Generation](https://mlsanity.com/paper/2503.22172/) (2025)
- [DreamReader: An Interpretability Toolkit for Text-to-Image Models](https://mlsanity.com/paper/2603.13299/) (2026)
- [Concept Removal for Frontier Image Generative Models](https://mlsanity.com/paper/2606.25548/) (2026)
- [Efficient bias mitigation in T2I diffusion models using Concept Graphs](https://mlsanity.com/paper/2607.03397/) (2026)
- [Concept Guidance: Precise, Training-Free Latent Control for Text-to-Image Generation](https://mlsanity.com/paper/2608.14172/) (2026)
- [Beyond the Linear Representation Hypothesis: Non-Linear Activation Steering in Text-to-Image Models](https://mlsanity.com/paper/2610.06945/) (2026)

## Abstract (arXiv)

Identifying layers within text-to-image models which control visual attributes can facilitate efficient model editing through closed-form updates. Recent work, leveraging causal tracing show that early Stable-Diffusion variants confine knowledge primarily to the first layer of the CLIP text-encoder, while it diffuses throughout the UNet.Extending this framework, we observe that for recent models (e.g., SD-XL, DeepFloyd), causal tracing fails in pinpointing localized knowledge, highlighting challenges in model editing. To address this issue, we introduce the concept of Mechanistic Localization in text-to-image models, where knowledge about various visual attributes (e.g., "style", "objects", "facts") can be mechanistically localized to a small fraction of layers in the UNet, thus facilitating efficient model editing. We localize knowledge using our method LocoGen which measures the direct effect of intermediate layers to output generation by performing interventions in the cross-attention layers of the UNet. We then employ LocoEdit, a fast closed-form editing method across popular open-source text-to-image models (including the latest SD-XL)and explore the possibilities of neuron-level model editing. Using Mechanistic Localization, our work offers a better view of successes and failures in localization-based text-to-image model editing. Code will be available at https://github.com/samyadeepbasu/LocoGen.