mlsanityData to 7 Oct 2026

Diffusion Models and Representation Learning: A Survey

arXiv 2407.00783 ↗First posted 30 Jun 2024cs.CV cs.AIcited by 53

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion Models are popular generative modeling methods in various vision tasks, attracting significant attention. They can be considered a unique instance of self-supervised learning methods due to their independence from label annotation. This survey explores the interplay between diffusion models and representation learning. It provides an overview of diffusion models' essential aspects, including mathematical foundations, popular denoising network architectures, and guidance methods. Various approaches related to diffusion models and representation learning are detailed. These include frameworks that leverage representations learned from pre-trained diffusion models for subsequent recognition tasks and methods that utilize advancements in representation and self-supervised learning to enhance diffusion models. This survey aims to offer a comprehensive overview of the taxonomy between diffusion models and representation learning, identifying key areas of existing concerns and potential exploration. Github link: https://github.com/dongzhuoyao/Diffusion-Representation-Learning-Survey-Taxonomy

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 167 of its 189 references that are in the map. First posted on arXiv 30 Jun 2024.

Builds on

167 of 189 references.

Closest ancestors · 55

Same direction, newest first

  1. 2024Multistep Distillation of Diffusion Models via Moment Matching6 Jun
  2. 2024Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance26 Mar
  3. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation18 Mar
  4. 2024Multistep Consistency Models11 Mar
  5. 2024Rethinking cluster-conditioned diffusion models for label-free image synthesis1 Mar
  6. 2024Scalable Diffusion Models with State Space Backbone8 Feb
  7. 2024Deconstructing Denoising Diffusion Models for Self-Supervised Learning25 Jan
  8. 2023Diffusion Model with Perceptual Loss30 Dec
  9. 2023Guided Diffusion from Self-Supervised Diffusion Features14 Dec
  10. 2023Boosting Latent Diffusion with Flow Matching12 Dec
  11. 2023Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models7 Dec
  12. 2023Diffusion Models Without Attention30 Nov
38 more
  1. 2023Do text-free diffusion models learn discriminative visual representations?29 Nov
  2. 2023SODA: Bottleneck Diffusion Models for Representation Learning29 Nov
  3. 2023State of the Art on Diffusion Models for Visual Computing11 Oct
  4. 2023Diffusion Model as Representation Learner21 Aug
  5. 2023Diffusion Models Beat GANs on Image Classification17 Jul
  6. 2023InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models14 Jun
  7. 2023ADDP: Learning General Representations for Image Recognition and Generation with Alternating Denoising Diffusion Process8 Jun
  8. 2023On the Design Fundamentals of Diffusion Models: A Survey7 Jun
  9. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models29 May
  10. 2023Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs6 May
  11. 2023Diffusion Models as Masked Autoencoders6 Apr
  12. 2023End-to-End Diffusion Latent Optimization Improves Classifier Guidance23 Mar
  13. 2023FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model17 Mar
  14. 2023Denoising Diffusion Autoencoders are Unified Self-supervised Learners17 Mar
  15. 2023One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale12 Mar
  16. 2023Consistency Models2 Mar
  17. 2023Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels21 Feb
  18. 2023Learning Data Representations with Joint Diffusion Models31 Jan
  19. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  20. 2022Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models26 Dec
  21. 2022Scalable Diffusion Models with Transformers19 Dec
  22. 2022Diffusion Models already have a Semantic Latent Space20 Oct
  23. 2022Self-Guided Diffusion Models12 Oct
  24. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidance3 Oct
  25. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  26. 2022Diffusion Models in Vision: A Survey10 Sep
  27. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  28. 2022Understanding Diffusion Models: A Unified Perspective25 Aug
  29. 2022Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model16 Aug
  30. 2022Classifier-Free Diffusion Guidance26 Jul
  31. 2022Diffusion models as plug-and-play priors17 Jun
  32. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  33. 2022Dynamic Dual-Output Diffusion Models8 Mar
  34. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  35. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representation30 Nov
  36. 2021Score-Based Generative Classifiers1 Oct
  37. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  38. 2021Variational Diffusion Models1 Jul

50 newest of 55 All 55 (JSON)

Other references · 112

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
38 more
  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2016Layer Normalizationcited by 5,579
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  4. 2022Flow Matching for Generative Modelingcited by 5,426
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2013Auto-Encoding Variational Bayescited by 3,980
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  11. 2020Longformer: The Long-Document Transformercited by 3,362
  12. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  13. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  14. 2016Wide Residual Networkscited by 3,057
  15. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  16. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  17. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  18. 2014Conditional Generative Adversarial Netscited by 2,152
  19. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  20. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  21. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  22. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  23. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  24. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  25. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  26. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  27. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  28. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
  29. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  30. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  31. 2019An Introduction to Variational Autoencoderscited by 997
  32. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  33. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  34. 2018Panoptic Segmentationcited by 796
  35. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 666
  36. 2019Panoptic Feature Pyramid Networkscited by 644
  37. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  38. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566

50 most cited of 112 All 112 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 25; 2026: 19.

53 papers.

Sort:

By area:

  1. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  2. 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 76
  3. 2024CleanDIFT: Diffusion Features without Noisecited by 42
  4. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesiscited by 42
  5. 2025Diff2Flow: Training Flow Matching Models via Diffusion Model Alignmentcited by 27
  6. 2025TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Trainingcited by 21
  7. 2024A Survey on Diffusion Models for Recommender Systemscited by 20
  8. 2024On the Feature Learning in Diffusion Modelscited by 19
  9. 2025Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modelingcited by 18
  10. 2024Distillation of Diffusion Features for Semantic Correspondencecited by 17
  11. 2025GenHancer: Imperfect Generative Models are Secretly Strong Vision-Centric Enhancerscited by 16
  12. 2024Exploring Low-Dimensional Subspaces in Diffusion Models for Controllable Image Editingcited by 15
38 more
  1. 2025From Image to Video: An Empirical Study of Diffusion Representationscited by 14
  2. 2025Adapting Self-Supervised Representations as a Latent Space for Efficient Generationcited by 13
  3. 2024Diffusion Models in 3D Vision: A Surveycited by 11
  4. 2024[MASK] is All You Needcited by 11
  5. 2025SARA: Structural and Adversarial Representation Alignment for Training-efficient Diffusion Modelscited by 11
  6. 2025Can Generative Geospatial Diffusion Models Excel as Discriminative Geospatial Foundation Models?cited by 10
  7. 2024Pixel-Space Post-Training of Latent Diffusion Modelscited by 6
  8. 2025Unleashing the Potential of Pre-Trained Diffusion Models for Generalizable Person Re-Identificationcited by 6
  9. 2025Guiding Registration with Emergent Similarity from Pre-Trained Diffusion Modelscited by 6
  10. 2025GECO: Geometrically Consistent Embedding with Lightspeed Inferencecited by 6
  11. 2025Efficient Generative Model Training via Embedded Representation Warmupcited by 5
  12. 2025DiffNMR: Diffusion Models for Nuclear Magnetic Resonance Spectra Elucidationcited by 5
  13. 2025SCFlow: Implicitly Learning Style and Content Disentanglement with Flow Modelscited by 5
  14. 2025SeaLion: Semantic Part-Aware Latent Point Diffusion Models for 3D Generationcited by 4
  15. 2026GenMask: Adapting DiT for Segmentation via Direct Mask Generationcited by 3
  16. 2026BindCLIP: A Unified Contrastive-Generative Representation Learning Framework for Virtual Screeningcited by 2
  17. 2026WorldCache: Content-Aware Caching for Accelerated Video World Modelscited by 2
  18. 2026Teacher-Feature Drifting: One-Step Diffusion Distillation with Pretrained Diffusion Representationscited by 2
  19. 2025MAP-based Problem-Agnostic diffusion model for Inverse Problemscited by 1
  20. 2025Contrastive Conditional-Unconditional Alignment for Long-tailed Diffusion Modelcited by 1
  21. 2025CVD-STORM: Cross-View Video Diffusion with Spatial-Temporal Reconstruction Model for Autonomous Drivingcited by 1
  22. 2026Diffusion Representations for Fine-Grained Image Classification: A Marine Plankton Case Studycited by 1
  23. 2026Evaluation without Generation: Non-Generative Assessment of Harmful Model Specialization with Applications to CSAMcited by 1
  24. 2026Evaluating the Representation Space of Diffusion Models via Self-Supervised Principlescited by 1
  25. 2026DiverseDiT++: Quantifying, Analyzing, and Promoting Representation Diversity in Diffusion Transformerscited by 1
  26. 2025Shape Modeling of Longitudinal Medical Images: From Diffeomorphic Metric Mapping to Deep Learningcited by 0
  27. 2025CLIP Meets Diffusion: A Synergistic Approach to Anomaly Detectioncited by 0
  28. 2025Latent Zoning Network: A Unified Principle for Generative Modeling, Representation Learning, and Classificationcited by 0
  29. 2025What We Don't C: Manifold Disentanglement for Structured Discoverycited by 0
  30. 2025Disentangled representations via score-based variational autoencoderscited by 0
  31. 2026Imitation from Observations with Trajectory-Level Generative Embeddingscited by 0
  32. 2026Self-Supervised Learning via Flow-Guided Neural Operator on Time-Series Datacited by 0
  33. 2026Representation learning from OCT imagescited by 0
  34. 2026Semantic Generative Tuning for Unified Multimodal Modelscited by 0
  35. 2026AME: A Multi-Type Contributor Attribution Framework in Generative AI Marketscited by 0
  36. 2026Learning a Maximum Entropy Model for Visual Textures using Diffusioncited by 0
  37. 2026Contrastive-Augmented Flow Matching for Style-Content Disentanglementcited by 0
  38. 2026Pixel-Space Diffusion Transformerscited by 0

50 most cited of 53 All 53 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion Models in Low-Level Vision: A Survey
  2. 2022Representation Learning with Diffusion Models
  3. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization
  4. 2025Learning Diffusion Models with Flexible Representation Guidance
  5. 2025Automated Learning of Semantic Embedding Representations for Diffusion Models
  6. 2022Efficient Diffusion Models for Vision: A Survey
  7. 2025Graph Representation Learning with Diffusion Generative Models
  8. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Models