mlsanityData to 7 Oct 2026

Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning

arXiv 2208.04202 ↗First posted 8 Aug 2022cs.CV cs.AI cs.CLcited by 362

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We present Bit Diffusion: a simple and generic approach for generating discrete data with continuous state and continuous time diffusion models. The main idea behind our approach is to first represent the discrete data as binary bits, and then train a continuous diffusion model to model these bits as real numbers which we call analog bits. To generate samples, the model first generates the analog bits, which are then thresholded to obtain the bits that represent the discrete variables. We further propose two simple techniques, namely Self-Conditioning and Asymmetric Time Intervals, which lead to a significant improvement in sample quality. Despite its simplicity, the proposed approach can achieve strong performance in both discrete image generation and image captioning tasks. For discrete image generation, we significantly improve previous state-of-the-art on both CIFAR-10 (which has 3K discrete 8-bit tokens) and ImageNet-64x64 (which has 12K discrete 8-bit tokens), outperforming the best autoregressive model in both sample quality (measured by FID) and efficiency. For image captioning on MS-COCO dataset, our approach achieves competitive results compared to autoregressive models.

Key facts. Cited by 362 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 57 of its 68 references that are in the map. First posted on arXiv 8 Aug 2022.

Builds on

57 of 68 references.

Closest ancestors · 11

Same direction, newest first

  1. 2022Classifier-Free Diffusion Guidance26 Jul
  2. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  3. 2022A Continuous Time Framework for Discrete Denoising Models30 May
  4. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  5. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  9. 2020Denoising Diffusion Implicit Models6 Oct
  10. 2020Denoising Diffusion Probabilistic Models19 Jun
  11. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  12. 2013Auto-Encoding Variational Bayescited by 3,980
34 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2015Variational Inference with Normalizing Flowscited by 2,161
  6. 2016Density estimation using Real NVPcited by 2,029
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  9. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  10. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  12. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  13. 2018Image Transformercited by 770
  14. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  15. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  16. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  17. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  18. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  19. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 345
  20. 2015A note on the evaluation of generative modelscited by 291
  21. 2022A Unified Sequence Interface for Vision Taskscited by 152
  22. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  23. 2018MaskGAN: Better Text Generation via Filling in the______cited by 89
  24. 2019Integer Discrete Flows and Lossless Compressioncited by 82
  25. 2019Discrete Flows: Invertible Generative Models of Discrete Datacited by 68
  26. 2019Latent Normalizing Flows for Discrete Sequencescited by 66
  27. 2017Maximum-Likelihood Augmented Discrete Generative Adversarial Networkscited by 39
  28. 2019Differentiable Product Quantization for End-to-End Embedding Compressioncited by 36
  29. 2020Categorical Normalizing Flows via Continuous Transformationscited by 36
  30. 2018On Self Modulation for Generative Adversarial Networkscited by 33
  31. 2017Boundary-Seeking Generative Adversarial Networkscited by 27
  32. 2019Perceptual Generative Autoencoderscited by 15
  33. 2018Learning K-way D-dimensional Discrete Codes for Compact Embedding Representationscited by 11
  34. 2021Discrete Denoising Flowscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 24; 2023: 83; 2024: 83; 2025: 91; 2026: 81.

362 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  2. 2025Large Language Diffusion Modelscited by 872
  3. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  4. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  5. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  6. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  7. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  8. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  9. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 373
  10. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  11. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  12. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
38 more
  1. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  2. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  3. 2024Discrete Flow Matchingcited by 301
  4. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  5. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  6. 2022A Survey on Generative Diffusion Modelcited by 238
  7. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 216
  8. 2023SE(3) diffusion model with application to protein backbone generationcited by 214
  9. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
  10. 2023A Survey on Video Diffusion Modelscited by 203
  11. 2022Continuous diffusion for categorical datacited by 202
  12. 2024Scaling up Masked Diffusion Models on Textcited by 195
  13. 2023On the Importance of Noise Scheduling for Diffusion Modelscited by 193
  14. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
  15. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  16. 2023One Transformer Fits All Distributions in Multi-Modal Diffusion at Scalecited by 185
  17. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
  18. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  19. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  20. 2023Likelihood-Based Diffusion Language Modelscited by 147
  21. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  22. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  23. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 129
  24. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  25. 2025The Diffusion Dualitycited by 124
  26. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  27. 2022Self-conditioned Embedding Diffusion for Text Generationcited by 114
  28. 2022Latent Diffusion for Language Generationcited by 110
  29. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 110
  30. 2024AlphaFold Meets Flow Matching for Generating Protein Ensemblescited by 105
  31. 2024FlowMM: Generating Materials with Riemannian Flow Matchingcited by 105
  32. 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
  33. 2024MaskBit: Embedding-free Image Generation via Bit Tokenscited by 96
  34. 2023DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuningcited by 82
  35. 2022Diffusion models for missing value imputation in tabular datacited by 77
  36. 2024Dual Diffusion for Unified Image Generation and Understandingcited by 76
  37. 2025A Survey on Diffusion Language Modelscited by 75
  38. 2023Dirichlet Diffusion Score Model for Biological Sequence Generationcited by 73

50 most cited of 362 All 362 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024BiDM: Pushing the Limit of Quantization for Diffusion Models
  2. 2024BitsFusion: 1.99 bits Weight Quantization of Diffusion Model
  3. 2025Discrete Diffusion Models for Language Generation
  4. 2024Think While You Generate: Discrete Diffusion with Planned Denoising
  5. 2026Generalized Discrete Diffusion from Snapshots
  6. 2025Compressed Image Generation with Denoising Diffusion Codebook Models
  7. 2025D2C: Unlocking the Potential of Continuous Autoregressive Image Generation with Discrete Tokens
  8. 2024DCTdiff: Intriguing Properties of Image Generative Modeling in the DCT Space