mlsanityData to 7 Oct 2026

Binary Latent Diffusion

arXiv 2304.04820 ↗First posted 10 Apr 2023cs.CVcited by 18

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

In this paper, we show that a binary latent space can be explored for compact yet expressive image representations. We model the bi-directional mappings between an image and the corresponding latent binary representation by training an auto-encoder with a Bernoulli encoding distribution. On the one hand, the binary latent space provides a compact discrete image representation of which the distribution can be modeled more efficiently than pixels or continuous latent representations. On the other hand, we now represent each image patch as a binary vector instead of an index of a learned cookbook as in discrete image representations with vector quantization. In this way, we obtain binary latent representations that allow for better image quality and high-resolution image representations without any multi-stage hierarchy in the latent space. In this binary latent space, images can now be generated effectively using a binary latent diffusion model tailored specifically for modeling the prior over the binary image representations. We present both conditional and unconditional image generation experiments with multiple datasets, and show that the proposed method performs comparably to state-of-the-art methods while dramatically improving the sampling efficiency to as few as 16 steps without using any test-time acceleration. The proposed framework can also be seamlessly scaled to 1024 × 1024 high-resolution image generation without resorting to latent hierarchy or multi-stage refinements.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 61 of its 78 references that are in the map. First posted on arXiv 10 Apr 2023.

Builds on

61 of 78 references.

Closest ancestors · 11

Same direction, newest first

  1. 2022Classifier-Free Diffusion Guidance26 Jul
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  3. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  4. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes24 Nov
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  9. 2020Denoising Diffusion Implicit Models6 Oct
  10. 2020Denoising Diffusion Probabilistic Models19 Jun
  11. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2017Neural Discrete Representation Learningcited by 4,650
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  11. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  12. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
38 more
  1. 2016Improved Techniques for Training GANscited by 2,938
  2. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  3. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  4. 2017Improved Training of Wasserstein GANscited by 2,223
  5. 2015Variational Inference with Normalizing Flowscited by 2,161
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  7. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  8. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  9. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  10. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  11. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  12. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  13. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  14. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  15. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  16. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  17. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  18. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  19. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 577
  20. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  21. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  22. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  23. 2021Generating Images with Sparse Representationscited by 262
  24. 2016Unrolled Generative Adversarial Networkscited by 208
  25. 2022Diffusion-GAN: Training GANs with Diffusioncited by 203
  26. 2022Diffusion-based Time Series Imputation and Forecasting with Structured State Space Modelscited by 195
  27. 2016A Theory of Generative ConvNetcited by 174
  28. 2021StyleSwin: Transformer-based GAN for High-resolution Image Generationcited by 152
  29. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  30. 2019Classification Accuracy Score for Conditional Generative Modelscited by 134
  31. 2020Improved Contrastive Divergence Training of Energy Based Modelscited by 127
  32. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  33. 2016Energy-based Generative Adversarial Networkcited by 119
  34. 2020Learning Latent Space Energy-Based Prior Modelcited by 104
  35. 2022Guided Diffusion Model for Adversarial Purificationcited by 90
  36. 2022Threat Model-Agnostic Adversarial Defense using Diffusion Modelscited by 24
  37. 2022Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Modelcited by 18
  38. 2020BinPlay: A Binary Latent Autoencoder for Generative Replay Continual Learningcited by 9

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 8; 2025: 7.

18 papers.

Sort:

By area:

  1. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  2. 2024SD-DiT: Unleashing the Power of Self-supervised Discrimination in Diffusion Transformercited by 47
  3. 2024BiDM: Pushing the Limit of Quantization for Diffusion Modelscited by 11
  4. 2024Cross-Domain Graph Data Scaling: A Showcase with Diffusion Modelscited by 8
  5. 2025CODA: Repurposing Continuous VAEs for Discrete Tokenizationcited by 8
  6. 2023Unaligned 2D to 3D Translation with Conditional Vector-Quantized Code Diffusion using Transformerscited by 7
  7. 2024Binary Noise for Binary Tasks: Masked Bernoulli Diffusion for Unsupervised Anomaly Detectioncited by 7
  8. 2023SDDPM: Speckle Denoising Diffusion Probabilistic Modelscited by 6
  9. 2024Elucidating the design space of language models for image generationcited by 6
  10. 2025Diffusion Model Quantization: A Reviewcited by 6
  11. 2024BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilitiescited by 5
  12. 2025BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super-Resolutioncited by 5
6 more
  1. 2025Instella-T2I: Pushing the Limits of 1D Discrete Latent Space Image Generationcited by 5
  2. 2024Conditional Text-to-Image Generation with Reference Guidancecited by 2
  3. 2025Adverse Weather Conditions Augmentation of LiDAR Scenes with Latent Diffusion Modelscited by 2
  4. 2024PearSAN: A Machine Learning Method for Inverse Design using Pearson Correlated Surrogate Annealingcited by 1
  5. 2025Binary Diffusion Probabilistic Modelcited by 1
  6. 2025On Designing Diffusion Autoencoders for Efficient Generation and Representation Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesis
  2. 2024Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion
  3. 2024Isometric Representation Learning for Disentangled Latent Space of Diffusion Models
  4. 2025Diffusion Transformers with Representation Autoencoders
  5. 2024Stabilize the Latent Space for Image Autoregressive Modeling: A Unified Perspective
  6. 2022Representation Learning with Diffusion Models
  7. 2022Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance
  8. 2024DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents