mlsanityData to 14 Oct 2026

Diffuse and Disperse: Image Generation with Representation Regularization

arXiv 2506.09027 ↗First posted 10 Jun 2025cs.CV cs.AI cs.LGcited by 70

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The development of diffusion-based generative models over the past decade has largely proceeded independently of progress in representation learning. These diffusion models typically rely on regression-based objectives and generally lack explicit regularization. In this work, we propose textitDispersive Loss, a simple plug-and-play regularizer that effectively improves diffusion-based generative models. Our loss function encourages internal representations to disperse in the hidden space, analogous to contrastive self-supervised learning, with the key distinction that it requires no positive sample pairs and therefore does not interfere with the sampling process used for regression. Compared to the recent method of representation alignment (REPA), our approach is self-contained and minimalist, requiring no pre-training, no additional parameters, and no external data. We evaluate Dispersive Loss on the ImageNet dataset across a range of models and report consistent improvements over widely used and strong baselines. We hope our work will help bridge the gap between generative modeling and representation learning.

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 39 of its 44 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

39 of 44 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025SARA: Structural and Adversarial Representation Alignment for Training-efficient Diffusion Models11 Mar
  2. 2025Inductive Moment Matching10 Mar
  3. 2024One Step Diffusion via Shortcut Models16 Oct
  4. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think9 Oct
  5. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers16 Jan
  6. 2023Improved Techniques for Training Consistency Models22 Oct
  7. 2022Scalable Diffusion Models with Transformers19 Dec
  8. 2022Classifier-Free Diffusion Guidance26 Jul
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  10. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  11. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  12. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
2 more
  1. 2020Denoising Diffusion Implicit Models6 Oct
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  10. 2022Flow Matching for Generative Modelingcited by 5,702
  11. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
13 more
  1. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  2. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  3. 2020Supervised Contrastive Learningcited by 2,829
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  5. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  6. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  7. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  8. 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
  9. 2025Mean Flows for One-step Generative Modelingcited by 561
  10. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 436
  11. 2024Flow Matching Guide and Codecited by 301
  12. 2022How Mask Matters: Towards Theoretical Understandings of Masked Autoencoderscited by 86
  13. 2025Aligning Text to Image in Diffusion Models is Easier Than You Thinkcited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 31; 2026: 39.

70 papers.

Sort:

By area:

  1. 2025Latent Diffusion Model without Variational Autoencodercited by 88
  2. 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 84
  3. 2026Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generationcited by 36
  4. 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
  5. 2026Self-Supervised Flow Matching for Scalable Multi-Modal Synthesiscited by 31
  6. 2026Improved Baselines with Representation Autoencoderscited by 31
  7. 2025One Layer Is Enough: Adapting Pretrained Visual Encoders for Image Generationcited by 28
  8. 2025X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learningcited by 25
  9. 2025From Editor to Dense Geometry Estimatorcited by 24
  10. 2025There is No VAE: End-to-End Pixel-Space Generative Modeling via Self-Supervised Pre-trainingcited by 24
  11. 2025Equilibrium Matching: Generative Modeling with Implicit Energy-Based Modelscited by 21
  12. 2025Decoupled MeanFlow: Turning Flow Models into Flow Maps for Accelerated Samplingcited by 20
38 more
  1. 2025Selective Underfitting in Diffusion Modelscited by 18
  2. 2025Guiding a Diffusion Transformer with the Internal Dynamics of Itselfcited by 16
  3. 2025Adversarial Flow Modelscited by 12
  4. 2026Stable Velocity: A Variance Perspective on Flow Matchingcited by 12
  5. 2025SuperFlow: Training Flow Matching Models with RL on the Flycited by 10
  6. 2025LayerSync: Self-aligning Intermediate Layerscited by 9
  7. 2025DM1: MeanFlow with Dispersive Regularization for 1-Step Robotic Manipulationcited by 8
  8. 2025A Survey on Long-Video Storytelling Generation: Architectures, Consistency, and Cinematic Qualitycited by 6
  9. 2025REAR: Rethinking Visual Autoregressive Models via Generator-Tokenizer Consistency Regularizationcited by 6
  10. 2025D2PPO: Diffusion Policy Policy Optimization with Dispersive Losscited by 5
  11. 2026DiverseDiT: Towards Diverse Representation Learning in Diffusion Transformerscited by 5
  12. 2026Drift-Based Policy Optimization: Native One-Step Policy Learning for Online Robot Controlcited by 5
  13. 2026Continuous Adversarial Flow Modelscited by 5
  14. 2025Cross-Architecture Distillation Made Simple with Redundancy Suppressioncited by 4
  15. 2025Sprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformerscited by 4
  16. 2025OMP: One-step Meanflow Policy with Directional Alignmentcited by 4
  17. 2026Boosting Latent Diffusion Models via Disentangled Representation Alignmentcited by 4
  18. 2025Flow Matching in the Low-Noise Regime: Pathologies and a Contrastive Remedycited by 3
  19. 2025Speedrunning ImageNet Diffusioncited by 3
  20. 2026Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generationcited by 3
  21. 2025MolSnap: Snap-Fast Molecular Generation with Latent Variational Mean Flowcited by 2
  22. 2026Dispersion Loss Counteracts Embedding Condensation and Improves Generalization in Small Language Modelscited by 2
  23. 2026MPDiT: Multi-Patch Global-to-Local Transformer Architecture For Efficient Flow Matching and Diffusion Modelcited by 2
  24. 2025Contrastive Conditional-Unconditional Alignment for Long-tailed Diffusion Modelcited by 1
  25. 2025Repulsor: Accelerating Generative Modeling with a Contrastive Memory Bankcited by 1
  26. 2025Group Diffusion: Enhancing Image Generation by Unlocking Cross-Sample Collaborationcited by 1
  27. 2025MixFlow Training: Alleviating Exposure Bias with Slowed Interpolation Mixturecited by 1
  28. 2026Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?cited by 1
  29. 2026BadRSSD: Backdoor Attacks on Regularized Self-Supervised Diffusion Modelscited by 1
  30. 2026Improving Diffusion Generalization with Weak-to-Strong Segmented Guidancecited by 1
  31. 2026InstanceRSR: Real-World Super-Resolution via Instance-Aware Representation Alignmentcited by 1
  32. 2026Generate in Reconstruction Space, Match in Semantic Space: Transport Geometry for One-Step Generationcited by 1
  33. 2026Continuous Language Diffusion as a Decoder-Interface Problemcited by 1
  34. 2026Don't Settle at the Mode! Mitigating Diversity Collapse in Pretrained Flow Models via Feature Self-Guidancecited by 1
  35. 2026Mitigating Compounding Error via Video Representation Regularizationcited by 1
  36. 2026UDT: Reconciling U-Nets and Diffusion Transformers with Data-Adaptive Token Reductioncited by 1
  37. 2026SPARE: Structural Parameter-Free Affinity Regularization for Flow Matchingcited by 1
  38. 2026DiverseDiT++: Quantifying, Analyzing, and Promoting Representation Diversity in Diffusion Transformerscited by 1

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Representation Learning with Diffusion Models
  2. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesis
  3. 2025Learning Diffusion Models with Flexible Representation Guidance
  4. 2022Image Embedding for Denoising Generative Models
  5. 2025Diffusion Transformers with Representation Autoencoders
  6. 2025No Alignment Needed for Generation: Learning Linearly Separable Representations in Diffusion Models
  7. 2024Diffusion Models Need Visual Priors for Image Generation
  8. 2026RPiAE: A Representation-Pivoted Autoencoder Enhancing Both Image Generation and Editing