mlsanityData to 7 Oct 2026

Return of Unconditional Generation: A Self-supervised Representation Generation Method

arXiv 2312.03701 ↗First posted 6 Dec 2023cs.CVcited by 95

Not on the map. Why?

Cited by 45 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Unconditional generation -- the problem of modeling data distribution without relying on human-annotated labels -- is a long-standing and fundamental challenge in generative models, creating a potential of learning from large-scale unlabeled data. In the literature, the generation quality of an unconditional method has been much worse than that of its conditional counterpart. This gap can be attributed to the lack of semantic information provided by labels. In this work, we show that one can close this gap by generating semantic representations in the representation space produced by a self-supervised encoder. These representations can be used to condition the image generator. This framework, called Representation-Conditioned Generation (RCG), provides an effective solution to the unconditional generation problem without using labels. Through comprehensive experiments, we observe that RCG significantly improves unconditional generation quality: e.g., it achieves a new state-of-the-art FID of 2.15 on ImageNet 256x256, largely reducing the previous best of 5.91 by a relative 64%. Our unconditional results are situated in the same tier as the leading class-conditional ones. We hope these encouraging observations will attract the community's attention to the fundamental problem of unconditional generation. Code is available at https://github.com/LTH14/rcg.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). Builds on 55 of its 70 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

55 of 70 references.

References · 55

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  12. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
38 more
  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2022Classifier-Free Diffusion Guidancecited by 5,717
  3. 2016Layer Normalizationcited by 5,579
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  10. 2016Improved Techniques for Training GANscited by 2,938
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  12. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  13. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  14. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  15. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  16. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  17. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  18. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  19. 2018Self-Attention Generative Adversarial Networkscited by 809
  20. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  21. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  22. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  23. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  24. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  25. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  26. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  27. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizercited by 260
  28. 2019Large Scale Adversarial Representation Learningcited by 238
  29. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 222
  30. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  31. 2022It's Raw! Audio Generation with State-Space Modelscited by 171
  32. 2021Instance-Conditioned GANcited by 119
  33. 2017Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Modelscited by 116
  34. 2016SampleRNN: An Unconditional End-to-End Neural Audio Generation Modelcited by 110
  35. 2023DiffiT: Diffusion Vision Transformers for Image Generationcited by 104
  36. 2021High Fidelity Visualization of What Your Self-Supervised Representation Knows Aboutcited by 64
  37. 2020Diverse Image Generation via Self-Conditioned GANscited by 54
  38. 2019High-Fidelity Image Generation With Fewer Labelscited by 51

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 11; 2025: 49; 2026: 34.

95 papers.

Sort:

By area:

  1. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  2. 2024The Platonic Representation Hypothesiscited by 362
  3. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  4. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
  5. 2024Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidancecited by 108
  6. 2025DDT: Decoupled Diffusion Transformercited by 100
  7. 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Thinkcited by 95
  8. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 93
  9. 2025Latent Diffusion Model without Variational Autoencodercited by 83
  10. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  11. 2025No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselvescited by 50
  12. 2026Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generationcited by 33
38 more
  1. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 28
  2. 2025"Principal Components" Enable A New Language of Imagescited by 25
  3. 2025There is No VAE: End-to-End Pixel-Space Generative Modeling via Self-Supervised Pre-trainingcited by 23
  4. 2025FlexVAR: Flexible Visual Autoregressive Modeling without Residual Predictioncited by 22
  5. 2025Highly Compressed Tokenizer Can Generate Without Trainingcited by 17
  6. 2025Semantics Lead the Way: Harmonizing Semantic and Texture Modeling with Asynchronous Latent Diffusioncited by 16
  7. 2024When Worse is Better: Navigating the compression-generation tradeoff in visual tokenizationcited by 14
  8. 2023Masked Modeling for Self-supervised Representation Learning on Vision and Beyondcited by 13
  9. 2025Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Modelscited by 13
  10. 2025Adapting Self-Supervised Representations as a Latent Space for Efficient Generationcited by 13
  11. 2025Learning Diffusion Models with Flexible Representation Guidancecited by 12
  12. 2025SVG-T2I: Scaling Up Text-to-Image Latent Diffusion Model Without Variational Autoencodercited by 12
  13. 2025Guiding a Diffusion Transformer with the Internal Dynamics of Itselfcited by 12
  14. 2025SARA: Structural and Adversarial Representation Alignment for Training-efficient Diffusion Modelscited by 11
  15. 2025PixCell: A generative foundation model for digital histopathology imagescited by 11
  16. 2025Scalable Training for Vector-Quantized Networks with 100% Codebook Utilizationcited by 10
  17. 2025Image Tokenizer Needs Post-Trainingcited by 10
  18. 2025Robust Latent Matters: Boosting Image Generation with Sampling Error Synthesiscited by 9
  19. 2025Autoregressive Distillation of Diffusion Transformerscited by 9
  20. 2025What Happens Next? Anticipating Future Motion by Generating Point Trajectoriescited by 9
  21. 2025MVAR: Visual Autoregressive Modeling with Scale and Spatial Markovian Conditioningcited by 8
  22. 2026SRA 2: Variational Autoencoder Self-Representation Alignment for Efficient Diffusion Trainingcited by 8
  23. 2026Learning a Generative Meta-Model of LLM Activationscited by 8
  24. 2024Learning Visual Generative Priors without Textcited by 7
  25. 2025Semantic Context Matters: Improving Conditioning for Autoregressive Modelscited by 7
  26. 2025Bidirectional Normalizing Flow: From Data to Noise and Backcited by 7
  27. 2025DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion Modelingcited by 6
  28. 2025SemanticGen: Video Generation in Semantic Spacecited by 6
  29. 2026VideoFlexTok: Flexible-Length Coarse-to-Fine Video Tokenizationcited by 6
  30. 2026Frequency-Aware Flow Matching for High-Quality Image Generationcited by 6
  31. 2025Efficient Generative Model Training via Embedded Representation Warmupcited by 5
  32. 2025Plug-and-Play Context Feature Reuse for Efficient Masked Generationcited by 5
  33. 2025Generative Audio Language Modeling with Continuous-valued Tokens and Masked Next-Token Predictioncited by 5
  34. 2025VFMF: World Modeling by Forecasting Vision Foundation Model Featurescited by 5
  35. 2026CaTok: Taming Mean Flows for One-Dimensional Causal Image Tokenizationcited by 5
  36. 2025HDCompression: Hybrid-Diffusion Image Compression for Ultra-Low Bitratescited by 4
  37. 2025FVAR: Visual Autoregressive Modeling via Next Focus Predictioncited by 4
  38. 2026DiverseDiT: Towards Diverse Representation Learning in Diffusion Transformerscited by 4

50 most cited of 95 All 95 (JSON)