mlsanityData to 7 Oct 2026

SODA: Bottleneck Diffusion Models for Representation Learning

arXiv 2311.17901 ↗First posted 29 Nov 2023cs.CV cs.AI cs.LGcited by 76

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We introduce SODA, a self-supervised diffusion model, designed for representation learning. The model incorporates an image encoder, which distills a source view into a compact representation, that, in turn, guides the generation of related novel views. We show that by imposing a tight bottleneck between the encoder and a denoising decoder, and leveraging novel view synthesis as a self-supervised objective, we can turn diffusion models into strong representation learners, capable of capturing visual semantics in an unsupervised manner. To the best of our knowledge, SODA is the first diffusion model to succeed at ImageNet linear-probe classification, and, at the same time, it accomplishes reconstruction, editing and synthesis tasks across a wide range of datasets. Further investigation reveals the disentangled nature of its emergent latent space, that serves as an effective interface to control and manipulate the model's produced images. All in all, we aim to shed light on the exciting and promising potential of diffusion models, not only for image generation, but also for learning rich and robust representations.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 91 of its 131 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

91 of 131 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models14 Jun
  2. 2023Denoising Diffusion Autoencoders are Unified Self-supervised Learners17 Mar
  3. 2022Classifier-Free Diffusion Guidance26 Jul
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representation30 Nov
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  8. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  9. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  10. 2020Denoising Diffusion Implicit Models6 Oct
  11. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 80

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  8. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
38 more
  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2014Going Deeper with Convolutionscited by 4,231
  9. 2013Auto-Encoding Variational Bayescited by 3,980
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  11. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  12. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  13. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  14. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  15. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  16. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  17. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  18. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  19. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  20. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  21. 2022Video Diffusion Modelscited by 2,280
  22. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  23. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  24. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  25. 2018Group Normalizationcited by 1,514
  26. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  27. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  28. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  29. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  30. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  31. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  32. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  33. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  34. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  35. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  36. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  37. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  38. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 26; 2025: 35; 2026: 15.

76 papers.

Sort:

By area:

  1. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  2. 2024Deconstructing Denoising Diffusion Models for Self-Supervised Learningcited by 89
  3. 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 76
  4. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  5. 2024Schr"odinger Bridge Flow for Unpaired Data Translationcited by 47
  6. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  7. 2024CleanDIFT: Diffusion Features without Noisecited by 42
  8. 2024Diffusion Feedback Helps CLIP See Bettercited by 37
  9. 2025Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenizationcited by 35
  10. 2024Diffusion Model with Cross Attention as an Inductive Bias for Disentanglementcited by 27
  11. 2024Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Trainingcited by 26
  12. 2025USP: Unified Self-Supervised Pretraining for Image Generation and Understandingcited by 24
38 more
  1. 2024UniGEM: A Unified Approach to Generation and Property Prediction for Moleculescited by 20
  2. 2024SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulationcited by 18
  3. 2025Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modelingcited by 18
  4. 2024Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentationcited by 16
  5. 2025GenHancer: Imperfect Generative Models are Secretly Strong Vision-Centric Enhancerscited by 16
  6. 2024MacDiff: Unified Skeleton Modeling with Masked Conditional Diffusioncited by 15
  7. 2024Image Understanding Makes for A Good Tokenizer for Image Generationcited by 14
  8. 2024Visual Lexicon: Rich Image Features in Language Spacecited by 12
  9. 2025Detect-and-Guide: Self-regulation of Diffusion Models for Safe Text-to-Image Generation via Guideline Token Optimizationcited by 12
  10. 2024Visual Representation Learning with Stochastic Frame Predictioncited by 10
  11. 2025D-AR: Diffusion via Autoregressive Modelscited by 9
  12. 2025MotionStrata: Hierarchical Motion Latents for Compact Video Autoencodingcited by 9
  13. 2024Divot: Diffusion Powers Video Tokenizer for Comprehension and Generationcited by 8
  14. 20254D Visual Pre-training for Robot Learningcited by 8
  15. 2024Understanding the Role of Equivariance in Self-supervised Learningcited by 7
  16. 2025SSDD: Single-Step Diffusion Decoder for Efficient Image Tokenizationcited by 7
  17. 2024Diffusion Bridge AutoEncoders for Unsupervised Representation Learningcited by 6
  18. 2025Unleashing the Potential of Pre-Trained Diffusion Models for Generalizable Person Re-Identificationcited by 6
  19. 2024An Analysis of Human Alignment of Latent Diffusion Modelscited by 5
  20. 2024Gen-SIS: Generative Self-augmentation Improves Self-supervised Learningcited by 4
  21. 2025SketchFusion: Learning Universal Sketch Features through Fusing Foundation Modelscited by 4
  22. 2025Latent Diffusion Autoencoders: Toward Efficient and Meaningful Unsupervised Representation Learning in Medical Imagingcited by 4
  23. 2025Vision-Language-Vision Auto-Encoder: Scalable Knowledge Distillation from Diffusion Modelscited by 4
  24. 2025REGEN: Learning Compact Video Embedding with (Re-)Generative Decodercited by 3
  25. 2025Can Diffusion Models Disentangle? A Theoretical Perspectivecited by 3
  26. 2025Conditioning Residuals for Diffusion Models via Representation Feedbackcited by 3
  27. 2025Generative Distribution Embeddings: Lifting autoencoders to the space of distributions for multiscale representation learningcited by 3
  28. 2025High-Performance Self-Supervised Learning by Joint Training of Flow Matchingcited by 3
  29. 2024Unified Auto-Encoding with Masked Diffusioncited by 2
  30. 2024Feature-guided score diffusion for sampling conditional densitiescited by 2
  31. 2024Nested Diffusion Models Using Hierarchical Latent Priorscited by 2
  32. 2025Sealing The Backdoor: Unlearning Adversarial Text Triggers In Diffusion Models Using Knowledge Distillationcited by 2
  33. 2025Bitrate-Controlled Diffusion for Disentangling Motion and Content in Videocited by 2
  34. 2025DIA: The Adversarial Exposure of Deterministic Inversion in Diffusion Modelscited by 2
  35. 2026BindCLIP: A Unified Contrastive-Generative Representation Learning Framework for Virtual Screeningcited by 2
  36. 2026StrADiff: A Structured Source-Wise Adaptive Diffusion Framework for Linear and Nonlinear Blind Source Separationcited by 2
  37. 2026Probabilistic Precipitation Nowcasting with Rectified Flow Transformerscited by 2
  38. 2024Inflationary Flows: Calibrated Bayesian Inference with Diffusion-Based Modelscited by 1

50 most cited of 76 All 76 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Representation Learning with Diffusion Models
  2. 2025Diffusion Transformers with Representation Autoencoders
  3. 2026Representation Learning in Diffusion and Flow-based Model: An Application Aspect
  4. 2025Generalization of Diffusion Models Arises with a Balanced Representation Space
  5. 2025Automated Learning of Semantic Embedding Representations for Diffusion Models
  6. 2025Learning Diffusion Models with Flexible Representation Guidance
  7. 2025Accelerating Diffusion Model Training under Minimal Budgets: A Condensation-Based Perspective
  8. 2026Coevolving Representations in Joint Image-Feature Diffusion