mlsanityData to 7 Oct 2026

Deconstructing Denoising Diffusion Models for Self-Supervised Learning

arXiv 2401.14404 ↗First posted 25 Jan 2024cs.CV cs.LGcited by 89

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

In this study, we examine the representation learning abilities of Denoising Diffusion Models (DDM) that were originally purposed for image generation. Our philosophy is to deconstruct a DDM, gradually transforming it into a classical Denoising Autoencoder (DAE). This deconstructive procedure allows us to explore how various components of modern DDMs influence self-supervised representation learning. We observe that only a very few modern components are critical for learning good representations, while many others are nonessential. Our study ultimately arrives at an approach that is highly simplified and to a large extent resembles a classical DAE. We hope our study will rekindle interest in a family of classical methods within the realm of modern self-supervised learning.

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 37 of its 49 references that are in the map. First posted on arXiv 25 Jan 2024.

Builds on

37 of 49 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023SODA: Bottleneck Diffusion Models for Representation Learning29 Nov
  2. 2023Diffusion Models Beat GANs on Image Classification17 Jul
  3. 2023Diffusion Models as Masked Autoencoders6 Apr
  4. 2023Denoising Diffusion Autoencoders are Unified Self-supervised Learners17 Mar
  5. 2022Scalable Diffusion Models with Transformers19 Dec
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  7. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  8. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  10. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  11. 2016Layer Normalizationcited by 5,579
  12. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
15 more
  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  5. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  6. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  7. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  8. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  9. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  10. 2016Deep Networks with Stochastic Depthcited by 973
  11. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  12. 2016Adversarial Feature Learningcited by 350
  13. 2019Large Scale Adversarial Representation Learningcited by 238
  14. 2022Corrupted Image Modeling for Self-Supervised Visual Pre-Trainingcited by 68
  15. 2023Mixed Autoencoder for Self-supervised Visual Representation Learningcited by 38

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 27; 2025: 37; 2026: 25.

89 papers.

Sort:

By area:

  1. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  2. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 125
  3. 2024Diffusion Language Models Are Versatile Protein Learnerscited by 93
  4. 2024Diffusion Actor-Critic with Entropy Regulatorcited by 81
  5. 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 76
  6. 2024Learning and Leveraging World Models in Visual Representation Learningcited by 71
  7. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  8. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributionscited by 53
  9. 2025No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselvescited by 50
  10. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  11. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 42
  12. 2026Image Generators are Generalist Vision Learnerscited by 40
38 more
  1. 2024VisionLLaMA: A Unified LLaMA Backbone for Vision Taskscited by 28
  2. 2025USP: Unified Self-Supervised Pretraining for Image Generation and Understandingcited by 24
  3. 2024UniGEM: A Unified Approach to Generation and Property Prediction for Moleculescited by 20
  4. 2025Decoupled MeanFlow: Turning Flow Models into Flow Maps for Accelerated Samplingcited by 19
  5. 2025Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modelingcited by 18
  6. 2025Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlocited by 18
  7. 2025Latent Denoising Makes Good Tokenizerscited by 18
  8. 2024Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentationcited by 16
  9. 2024DP-IQA: Utilizing Diffusion Prior for Blind Image Quality Assessment in the Wildcited by 14
  10. 2024TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representationcited by 14
  11. 2024Frozen-DETR: Enhancing DETR with Image Understanding from Frozen Foundation Modelscited by 13
  12. 2025Enhanced DACER Algorithm with High Diffusion Efficiencycited by 13
  13. 2025Learning Diffusion Models with Flexible Representation Guidancecited by 12
  14. 2024GenRec: Unifying Video Generation and Recognition with Diffusion Modelscited by 11
  15. 2024Diffusion Model with Representation Alignment for Protein Inverse Foldingcited by 10
  16. 2024Masked Completion via Structured Diffusion with White-Box Transformerscited by 9
  17. 2024Zero-Shot Video Restoration and Enhancement Using Pre-Trained Image Diffusion Modelcited by 8
  18. 2026Making Foresight Actionable: Repurposing Representation Alignment in World Action Modelscited by 8
  19. 2024SIDE: Surrogate Conditional Data Extraction from Diffusion Modelscited by 7
  20. 2024Optical Flow Representation Alignment Mamba Diffusion Model for Medical Video Generationcited by 7
  21. 2025No Alignment Needed for Generation: Learning Linearly Separable Representations in Diffusion Modelscited by 7
  22. 2025Generative Classifiers Avoid Shortcut Solutionscited by 7
  23. 2025Guiding Registration with Emergent Similarity from Pre-Trained Diffusion Modelscited by 6
  24. 2024Diffusion Features to Bridge Domain Gap for Semantic Segmentationcited by 5
  25. 2024BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilitiescited by 5
  26. 2025Diffusion Models for Computational Neuroimaging: A Surveycited by 5
  27. 2025UniWav: Towards Unified Pre-training for Speech Representation Learning and Generationcited by 5
  28. 2025Efficient Generative Model Training via Embedded Representation Warmupcited by 5
  29. 2025Learning Actionable World Models for Industrial Process Controlcited by 4
  30. 2025Vision-Language-Vision Auto-Encoder: Scalable Knowledge Distillation from Diffusion Modelscited by 4
  31. 2025Diff-ICMH: Harmonizing Machine and Human Vision in Image Compression with Generative Priorcited by 4
  32. 2025Flow Along the K-Amplitude for Generative Modelingcited by 3
  33. 2025Conditioning Residuals for Diffusion Models via Representation Feedbackcited by 3
  34. 2025Towards Multimodal Understanding via Stable Diffusion as a Task-Aware Feature Extractorcited by 3
  35. 2025High-Performance Self-Supervised Learning by Joint Training of Flow Matchingcited by 3
  36. 2024Zippo: Zipping Color and Transparency Distributions into a Single Diffusion Modelcited by 2
  37. 2024Unified Auto-Encoding with Masked Diffusioncited by 2
  38. 2025From Pixels to Components: Eigenvector Masking for Visual Representation Learningcited by 2

50 most cited of 89 All 89 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Representation Learning with Diffusion Models
  2. 2025Generalization of Diffusion Models Arises with a Balanced Representation Space
  3. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representation
  4. 2026Should We Skip Diffusion?
  5. 2022Image Embedding for Denoising Generative Models
  6. 2026LDM-is-AE: Latent Diffusion Model is an Auto-Encoder for End-to-End Image Generation
  7. 2025Diffusion Transformers with Representation Autoencoders
  8. 2025On Designing Diffusion Autoencoders for Efficient Generation and Representation Learning