# NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models

arXiv: https://arxiv.org/abs/2312.05390 · HTML: https://mlsanity.com/paper/2312.05390/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 36 of its 46 references that are in the map.
- First posted on arXiv 8 Dec 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs](https://mlsanity.com/paper/2005.09635/) (2020)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Diffusion Models already have a Semantic Latent Space](https://mlsanity.com/paper/2210.10960/) (2022)
- [Unsupervised Discovery of Interpretable Directions in the GAN Latent Space](https://mlsanity.com/paper/2002.03754/) (2020)
- [DeepFakes: a New Threat to Face Recognition? Assessment and Detection](https://mlsanity.com/paper/1812.08685/) (2018)
- [GANalyze: Toward Visual Definitions of Cognitive Image Properties](https://mlsanity.com/paper/1906.10112/) (2019)
- [StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners](https://mlsanity.com/paper/2306.00984/) (2023)
- [Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models](https://mlsanity.com/paper/2311.12092/) (2023)
- [Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry](https://mlsanity.com/paper/2307.12868/) (2023)
- [Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.08698/) (2022)
- [Deep Feature Interpolation for Image Content Changes](https://mlsanity.com/paper/1611.05507/) (2016)
- [Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models](https://mlsanity.com/paper/2303.11073/) (2023)

## Built on by (most cited first, 30 of 47)

- [Interpreting the Weight Space of Customized Diffusion Models](https://mlsanity.com/paper/2406.09413/) (2024)
- [FluxSpace: Disentangled Semantic Editing in Rectified Flow Transformers](https://mlsanity.com/paper/2412.09611/) (2024)
- [SliderSpace: Decomposing the Visual Capabilities of Diffusion Models](https://mlsanity.com/paper/2502.01639/) (2025)
- [DreamRelation: Relation-Centric Video Customization](https://mlsanity.com/paper/2503.07602/) (2025)
- [SuperEdit: Rectifying and Facilitating Supervision for Instruction-Based Image Editing](https://mlsanity.com/paper/2505.02370/) (2025)
- [DiffusionPID: Interpreting Diffusion via Partial Information Decomposition](https://mlsanity.com/paper/2406.05191/) (2024)
- [Exploring Low-Dimensional Subspaces in Diffusion Models for Controllable Image Editing](https://mlsanity.com/paper/2409.02374/) (2024)
- [LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformers](https://mlsanity.com/paper/2505.23758/) (2025)
- [Reference-Based 3D-Aware Image Editing with Triplanes](https://mlsanity.com/paper/2404.03632/) (2024)
- [Learn Once, Edit Anywhere: Visual Direction Transfer for Diffusion Models](https://mlsanity.com/paper/2403.19645/) (2024)
- [Text Slider: Efficient and Plug-and-Play Continuous Concept Control for Image/Video Synthesis via LoRA Adapters](https://mlsanity.com/paper/2509.18831/) (2025)
- [The Curious Case of End Token: A Zero-Shot Disentangled Image Editing using CLIP](https://mlsanity.com/paper/2406.00457/) (2024)
- [HOFAR: High-Order Augmentation of Flow Autoregressive Transformers](https://mlsanity.com/paper/2503.08032/) (2025)
- [FreeSliders: Training-Free, Modality-Agnostic Concept Sliders for Fine-Grained Diffusion Control in Images, Audio, and Video](https://mlsanity.com/paper/2511.00103/) (2025)
- [UniLight: A Unified Representation for Lighting](https://mlsanity.com/paper/2512.04267/) (2025)
- [Explaining in Diffusion: Explaining a Classifier Through Hierarchical Semantics with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2412.18604/) (2024)
- [From Cradle to Cane: A Two-Pass Framework for High-Fidelity Lifespan Face Aging](https://mlsanity.com/paper/2506.20977/) (2025)
- [Fully Unsupervised Self-debiasing of Text-to-Image Diffusion Models](https://mlsanity.com/paper/2512.03749/) (2025)
- [The Universal Normal Embedding](https://mlsanity.com/paper/2603.21786/) (2026)
- [Mechanistically Eliciting Latent Behaviors in Language Models](https://mlsanity.com/paper/2606.29604/) (2026)
- [Exploring Structured Semantic Priors Underlying Diffusion Score for Test-time Adaptation](https://mlsanity.com/paper/2501.00873/) (2025)
- [An h-space Based Adversarial Attack for Protection Against Few-shot Personalization](https://mlsanity.com/paper/2507.17554/) (2025)
- [Contrastive Diffusion Alignment: Learning Structured Latents for Controllable Generation](https://mlsanity.com/paper/2510.14190/) (2025)
- [SCALEX: Scalable Concept and Latent Exploration for Diffusion Models](https://mlsanity.com/paper/2511.13750/) (2025)
- [ELROND: Exploring and decomposing intrinsic capabilities of diffusion models](https://mlsanity.com/paper/2602.10216/) (2026)
- [DiReCT: Disentangled Regularization of Contrastive Trajectories for Physics-Refined Video Generation](https://mlsanity.com/paper/2603.25931/) (2026)
- [Decoding Diffusion: A Scalable Framework for Unsupervised Analysis of Latent Space Biases and Representations Using Natural Language Prompts](https://mlsanity.com/paper/2410.21314/) (2024)
- [Unsupervised Region-Based Image Editing of Denoising Diffusion Models](https://mlsanity.com/paper/2412.12912/) (2024)
- [Instructing Text-to-Image Diffusion Models via Classifier-Guided Semantic Optimization](https://mlsanity.com/paper/2505.14254/) (2025)
- [Contrastive Conditional-Unconditional Alignment for Long-tailed Diffusion Model](https://mlsanity.com/paper/2507.09052/) (2025)

## Abstract (arXiv)

Generative models have been very popular in the recent years for their image generation capabilities. GAN-based models are highly regarded for their disentangled latent space, which is a key feature contributing to their success in controlled image editing. On the other hand, diffusion models have emerged as powerful tools for generating high-quality images. However, the latent space of diffusion models is not as thoroughly explored or understood. Existing methods that aim to explore the latent space of diffusion models usually relies on text prompts to pinpoint specific semantics. However, this approach may be restrictive in areas such as art, fashion, or specialized fields like medicine, where suitable text prompts might not be available or easy to conceive thus limiting the scope of existing work. In this paper, we propose an unsupervised method to discover latent semantics in text-to-image diffusion models without relying on text prompts. Our method takes a small set of unlabeled images from specific domains, such as faces or cats, and a pre-trained diffusion model, and discovers diverse semantics in unsupervised fashion using a contrastive learning objective. Moreover, the learned directions can be applied simultaneously, either within the same domain (such as various types of facial edits) or across different domains (such as applying cat and face edits within the same image) without interfering with each other. Our extensive experiments show that our method achieves highly disentangled edits, outperforming existing approaches in both diffusion-based and GAN-based latent space editing methods.