mlsanityData to 14 Oct 2026

Latent Diffusion U-Net Representations Contain Positional Embeddings and Anomalies

arXiv 2504.07008 ↗First posted 9 Apr 2025cs.CVcited by 1

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have demonstrated remarkable capabilities in synthesizing realistic images, spurring interest in using their representations for various downstream tasks. To better understand the robustness of these representations, we analyze popular Stable Diffusion models using representational similarity and norms. Our findings reveal three phenomena: (1) the presence of a learned positional embedding in intermediate representations, (2) high-similarity corner artifacts, and (3) anomalous high-norm artifacts. These findings underscore the need to further investigate the properties of diffusion model representations before considering them for downstream tasks that require robust features. Project page: https://jonasloos.github.io/sd-representation-anomalies

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 44 of its 47 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

44 of 47 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024CleanDIFT: Diffusion Features without Noise4 Dec
  2. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation18 Mar
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  4. 2024Deconstructing Denoising Diffusion Models for Self-Supervised Learning25 Jan
  5. 2023SODA: Bottleneck Diffusion Models for Representation Learning29 Nov
  6. 2023Adversarial Diffusion Distillation28 Nov
  7. 2023Diffusion Models Beat GANs on Image Classification17 Jul
  8. 2023Denoising Diffusion Autoencoders are Unified Self-supervised Learners17 Mar
  9. 2023Unsupervised Discovery of Semantic Latent Directions in Diffusion Models24 Feb
  10. 2022Scalable Diffusion Models with Transformers19 Dec
  11. 2022Diffusion Models already have a Semantic Latent Space20 Oct
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
2 more
  1. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2022Generative Adversarial Networkscited by 5,627
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2023Vision Transformers Need Registerscited by 871
  10. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 499
  11. 2023Emergent Correspondence from Image Diffusioncited by 446
  12. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 327
18 more
  1. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 301
  2. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 269
  3. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 214
  4. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 177
  5. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignmentcited by 171
  6. 2023Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusioncited by 131
  7. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  8. 2023Telling Left from Right: Identifying Geometry-Aware Semantic Correspondencecited by 86
  9. 2023Intriguing properties of generative classifierscited by 52
  10. 2024ECoDepth: Effective Conditioning of Diffusion Models for Monocular Depth Estimationcited by 45
  11. 2023SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matchingcited by 38
  12. 2023Improving Semantic Correspondence with Viewpoint-Guided Spherical Mapscited by 34
  13. 2023Beyond Surface Statistics: Scene Representations in a Latent Diffusion Modelcited by 30
  14. 2024DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cutcited by 26
  15. 2024Distillation of Diffusion Features for Semantic Correspondencecited by 17
  16. 2024An Analysis of Human Alignment of Latent Diffusion Modelscited by 5
  17. 2024Diffusion Features to Bridge Domain Gap for Semantic Segmentationcited by 5
  18. 2023Match me if you can: Semi-Supervised Semantic Correspondence Learning with Unpaired Imagescited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Diffusion model for gradient preconditioning in hyperspectral imaging inverse problemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusing in the Right Space: A Systematic Study of Latent Diffusability
  2. 2025No Alignment Needed for Generation: Learning Linearly Separable Representations in Diffusion Models
  3. 2024Do Diffusion Models Learn Semantically Meaningful and Efficient Representations?
  4. 2025Evaluating Robustness in Latent Diffusion Models via Embedding Level Augmentation
  5. 2024Diffusion Models Need Visual Priors for Image Generation
  6. 2024How Diffusion Models Learn to Factorize and Compose
  7. 2026Purify Before You Align: Improving Representation Alignment for Diffusion Models
  8. 2022Image Embedding for Denoising Generative Models