mlsanityData to 7 Oct 2026

Representation Learning in Diffusion and Flow-based Model: An Application Aspect

arXiv 2608.24068 ↗First posted 25 Aug 2026cs.CVcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models and flow-based models have recently become the dominant paradigms in generative modeling, largely due to their ability to learn rich, multi-level visual representations through large-scale training. This creates a bidirectional relationship between generative models and representation learning: improving representation learning enhances generation quality, while the learned representations can be leveraged for broader understanding tasks. This survey systematically explores this interplay with a focus on applications. We propose a three-tier progressive framework that organizes existing works from three perspectives: using representation learning to improve generative capabilities, exploiting generative models to extract representations for perception tasks, and ultimately moving toward general-purpose unified applications. We systematically categorize representative methods across a wide range of downstream tasks, including image classification, dense visual prediction, instance-level perception, and annotation-scarce scenarios. By providing a unified taxonomy and identifying key challenges, this survey aims to clarify the underlying logic of current research and suggest promising directions for future exploration. We hope this work can serve as a valuable reference for researchers interested in harnessing the representation power of generative models for applications beyond generation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 122 of its 153 references that are in the map. First posted on arXiv 25 Aug 2026.

Builds on

122 of 153 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusion8 May
  2. 2026Test-Time Conditioning with Representation-Aligned Visual Features3 Feb
  3. 2026Hyperspherical Autoencoder for High-Fidelity Image Reconstruction and Generation30 Jan
  4. 2026Diffusion Representations for Fine-Grained Image Classification: A Marine Plankton Case Study19 Jan
  5. 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think2 Jul
  6. 2025Diffuse and Disperse: Image Generation with Representation Regularization10 Jun
  7. 2025REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training22 May
  8. 2025U-REPA: Aligning Diffusion U-Nets to ViTs24 Mar
  9. 2024DiFiC: Your Diffusion Model Holds the Secret to Fine-Grained Clustering25 Dec
  10. 2024Diffusion Models and Representation Learning: A Survey30 Jun
  11. 2023Diffusion Models Beat GANs on Image Classification17 Jul
  12. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 110

  1. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  2. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  3. 2025DINOv3cited by 1,297
  4. 2025Large Language Diffusion Modelscited by 872
  5. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  6. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  7. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  8. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  9. 2023Zero-shot Image-to-Image Translationcited by 503
  10. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  11. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  12. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
38 more
  1. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  2. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  3. 2017Data Augmentation Generative Adversarial Networkscited by 278
  4. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  5. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
  6. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 233
  7. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 218
  8. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
  9. 2023Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robustcited by 179
  10. 2023StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learnerscited by 177
  11. 2023Planting a SEED of Vision in Large Language Modelcited by 140
  12. 2023Invisible Image Watermarks Are Provably Removable Using Generative AIcited by 124
  13. 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generationcited by 120
  14. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  15. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 108
  16. 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 105
  17. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  18. 2023Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentationcited by 100
  19. 2023Return of Unconditional Generation: A Self-supervised Representation Generation Methodcited by 95
  20. 2023Mist: Towards Improved Adversarial Examples for Diffusion Modelscited by 91
  21. 2018BAGAN: Data Augmentation with Balancing GANcited by 90
  22. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 88
  23. 2024MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizercited by 80
  24. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 76
  25. 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Modelscited by 68
  26. 2025WorldEval: World Model as Real-World Robot Policies Evaluatorcited by 64
  27. 2023FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Modelscited by 61
  28. 2023A Reproducible Extraction of Training Images from Diffusion Modelscited by 60
  29. 2025UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokenscited by 55
  30. 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Modelscited by 54
  31. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 53
  32. 2025Unified Autoregressive Visual Generation and Understanding with Continuous Tokenscited by 51
  33. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  34. 2022UniTune: Text-Driven Image Editing by Fine Tuning a Diffusion Model on a Single Imagecited by 42
  35. 2022Positive-incentive Noisecited by 39
  36. 2023Variational Positive-incentive Noise: How Noise Benefits Modelscited by 38
  37. 2025Skywork UniPic 2.0: Building Kontext Model with Online RL for Unified Multimodal Modelcited by 38
  38. 2025LightLab: Controlling Light Sources in Images with Diffusion Modelscited by 36

50 most cited of 110 All 110 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025From Image to Video: An Empirical Study of Diffusion Representations
  2. 2023Do text-free diffusion models learn discriminative visual representations?
  3. 2024Diffusion Models in Low-Level Vision: A Survey
  4. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesis
  5. 2024Bridging Generative and Discriminative Models for Unified Visual Perception with Diffusion Priors
  6. 2023State of the Art on Diffusion Models for Visual Computing
  7. 2024Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Models
  8. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think