Representation Learning in Diffusion and Flow-based Model: An Application Aspect
On the map: Diffusion models → Diffusion foundations and sampling
Abstract (arXiv)
Diffusion models and flow-based models have recently become the dominant paradigms in generative modeling, largely due to their ability to learn rich, multi-level visual representations through large-scale training. This creates a bidirectional relationship between generative models and representation learning: improving representation learning enhances generation quality, while the learned representations can be leveraged for broader understanding tasks. This survey systematically explores this interplay with a focus on applications. We propose a three-tier progressive framework that organizes existing works from three perspectives: using representation learning to improve generative capabilities, exploiting generative models to extract representations for perception tasks, and ultimately moving toward general-purpose unified applications. We systematically categorize representative methods across a wide range of downstream tasks, including image classification, dense visual prediction, instance-level perception, and annotation-scarce scenarios. By providing a unified taxonomy and identifying key challenges, this survey aims to clarify the underlying logic of current research and suggest promising directions for future exploration. We hope this work can serve as a valuable reference for researchers interested in harnessing the representation power of generative models for applications beyond generation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 122 of its 153 references that are in the map. First posted on arXiv 25 Aug 2026.
Builds on
122 of 153 references.
Closest ancestors · 12
Same direction, newest first
- 2026What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusion8 May
- 2026Test-Time Conditioning with Representation-Aligned Visual Features3 Feb
- 2026Hyperspherical Autoencoder for High-Fidelity Image Reconstruction and Generation30 Jan
- 2026Diffusion Representations for Fine-Grained Image Classification: A Marine Plankton Case Study19 Jan
- 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think2 Jul
- 2025Diffuse and Disperse: Image Generation with Representation Regularization10 Jun
- 2025REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training22 May
- 2025U-REPA: Aligning Diffusion U-Nets to ViTs24 Mar
- 2024DiFiC: Your Diffusion Model Holds the Secret to Fine-Grained Clustering25 Dec
- 2024Diffusion Models and Representation Learning: A Survey30 Jun
- 2023Diffusion Models Beat GANs on Image Classification17 Jul
- 2020Denoising Diffusion Probabilistic Models19 Jun
Other references · 110
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
- 2025DINOv3cited by 1,297
- 2025Large Language Diffusion Modelscited by 872
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2025Dream 7B: Diffusion Large Language Modelscited by 515
- 2023Zero-shot Image-to-Image Translationcited by 503
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
38 more
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
- 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
- 2017Data Augmentation Generative Adversarial Networkscited by 278
- 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
- 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 233
- 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 218
- 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
- 2023Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robustcited by 179
- 2023StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learnerscited by 177
- 2023Planting a SEED of Vision in Large Language Modelcited by 140
- 2023Invisible Image Watermarks Are Provably Removable Using Generative AIcited by 124
- 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generationcited by 120
- 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
- 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 108
- 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 105
- 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
- 2023Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentationcited by 100
- 2023Return of Unconditional Generation: A Self-supervised Representation Generation Methodcited by 95
- 2023Mist: Towards Improved Adversarial Examples for Diffusion Modelscited by 91
- 2018BAGAN: Data Augmentation with Balancing GANcited by 90
- 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 88
- 2024MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizercited by 80
- 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 76
- 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Modelscited by 68
- 2025WorldEval: World Model as Real-World Robot Policies Evaluatorcited by 64
- 2023FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Modelscited by 61
- 2023A Reproducible Extraction of Training Images from Diffusion Modelscited by 60
- 2025UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokenscited by 55
- 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Modelscited by 54
- 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 53
- 2025Unified Autoregressive Visual Generation and Understanding with Continuous Tokenscited by 51
- 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
- 2022UniTune: Text-Driven Image Editing by Fine Tuning a Diffusion Model on a Single Imagecited by 42
- 2022Positive-incentive Noisecited by 39
- 2023Variational Positive-incentive Noise: How Noise Benefits Modelscited by 38
- 2025Skywork UniPic 2.0: Building Kontext Model with Online RL for Unified Multimodal Modelcited by 38
- 2025LightLab: Controlling Light Sources in Images with Diffusion Modelscited by 36
50 most cited of 110 All 110 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025From Image to Video: An Empirical Study of Diffusion Representations
- 2023Do text-free diffusion models learn discriminative visual representations?
- 2024Diffusion Models in Low-Level Vision: A Survey
- 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesis
- 2024Bridging Generative and Discriminative Models for Unified Visual Perception with Diffusion Priors
- 2023State of the Art on Diffusion Models for Visual Computing
- 2024Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Models
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think