mlsanityData to 14 Oct 2026

Improved Vector Quantized Diffusion Models

arXiv 2205.16007 ↗First posted 31 May 2022cs.CVcited by 62

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Vector quantized diffusion (VQ-Diffusion) is a powerful generative model for text-to-image synthesis, but sometimes can still generate low-quality samples or weakly correlated images with text input. We find these issues are mainly due to the flawed sampling strategy. In this paper, we propose two important techniques to further improve the sample quality of VQ-Diffusion. 1) We explore classifier-free guidance sampling for discrete denoising diffusion model and propose a more general and effective implementation of classifier-free guidance. 2) We present a high-quality inference strategy to alleviate the joint distribution issue in VQ-Diffusion. Finally, we conduct experiments on various datasets to validate their effectiveness and show that the improved VQ-Diffusion suppresses the vanilla version by large margins. We achieve an 8.44 FID score on MSCOCO, surpassing VQ-Diffusion by 5.42 FID score. When trained on ImageNet, we dramatically improve the FID score from 11.89 to 4.83, demonstrating the superiority of our proposed techniques.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 38 of its 45 references that are in the map. First posted on arXiv 31 May 2022.

Builds on

38 of 45 references.

Closest ancestors · 8

Same direction, newest first

  1. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  2. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  3. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching5 Jun
  4. 2021On Fast Sampling of Diffusion Probabilistic Models31 May
  5. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  8. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  9. 2022Video Diffusion Modelscited by 2,305
  10. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  11. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  12. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
18 more
  1. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  2. 2016Generative Adversarial Text to Image Synthesiscited by 735
  3. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  4. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  5. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  6. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  7. 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 137
  8. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  9. 2021M6: A Chinese Multimodal Pretrainercited by 91
  10. 2018Photographic Text-to-Image Synthesis with a Hierarchically-nested Adversarial Networkcited by 64
  11. 2018Tell, Draw, and Repeat: Generating and Modifying Images Based on Continual Linguistic Instructioncited by 61
  12. 2019Mask-Guided Portrait Editing with Conditional GANscited by 61
  13. 2020GIQA: Generated Image Quality Assessmentcited by 51
  14. 2021DAE-GAN: Dynamic Aspect-aware GAN for Text-to-Image Synthesiscited by 45
  15. 2017TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Networkcited by 21
  16. 2019Dual Adversarial Inference for Text-to-Image Synthesiscited by 17
  17. 2020Efficient Neural Architecture for Text-to-Image Synthesiscited by 11
  18. 2020PriorGAN: Real Data Prior for Generative Adversarial Netscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 12; 2023: 22; 2024: 20; 2025: 5; 2026: 3.

62 papers.

Sort:

By area:

  1. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  2. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
  3. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  4. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 305
  5. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 265
  6. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 246
  7. 2022A Survey on Generative Diffusion Modelcited by 240
  8. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidancecited by 154
  9. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  10. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 109
  11. 2022Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesiscited by 75
  12. 2024Scaling the Codebook Size of VQGAN to 100,000 with a Utilization Rate of 99%cited by 68
38 more
  1. 2023Towards Accurate Image Coding: Improved Autoregressive Image Generation with Dynamic Vector Quantizationcited by 60
  2. 2022Efficient Diffusion Models for Vision: A Surveycited by 51
  3. 2024Distillation of Discrete Diffusion through Dimensional Correlationscited by 48
  4. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  5. 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Modelscited by 41
  6. 2022Unified Discrete Diffusion for Simultaneous Vision-Language Generationcited by 39
  7. 2022Shifted Diffusion for Text-to-image Generationcited by 37
  8. 2023Efficient and Scalable Graph Generation through Iterative Local Expansioncited by 36
  9. 2023Multi-Architecture Multi-Expert Diffusion Modelscited by 35
  10. 2023Efficient-VQGAN: Towards High-Resolution Image Generation with Efficient Vision Transformerscited by 33
  11. 2024Improving Diffusion-Based Image Synthesis with Context Predictioncited by 32
  12. 2024Cometh: A continuous-time discrete-state graph diffusion modelcited by 32
  13. 2022Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesiscited by 31
  14. 2023LD-ZNet: A Latent Diffusion Approach for Text-Based Image Segmentationcited by 29
  15. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
  16. 20233DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Processcited by 25
  17. 2022Vision+X: A Survey on Multimodal Learning in the Light of Datacited by 22
  18. 2023Long-Term Rhythmic Video Soundtrackercited by 17
  19. 2022Exploring Discrete Diffusion Models for Image Captioningcited by 16
  20. 2023Diffusion Models in Bioinformatics: A New Wave of Deep Learning Revolution in Actioncited by 15
  21. 2023Classifier-free graph diffusion for molecular property targetingcited by 14
  22. 2023GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Frameworkcited by 13
  23. 2025Uniform Discrete Diffusion with Metric Path for Video Generationcited by 12
  24. 2024[MASK] is All You Needcited by 11
  25. 2023StraIT: Non-autoregressive Generation with Stratified Image Transformercited by 10
  26. 2024Text-guided Explorable Image Super-resolutioncited by 9
  27. 2023Spiking-Diffusion: Vector Quantized Discrete Diffusion Model with Spiking Neural Networkscited by 7
  28. 2023Text-Conditioned Sampling Framework for Text-to-Image Generation with Masked Generative Modelscited by 6
  29. 2023Using External Off-Policy Speech-To-Text Mappings in Contextual End-To-End Automated Speech Recognitioncited by 5
  30. 2023Fast non-autoregressive inverse folding with discrete diffusioncited by 5
  31. 2024Vector Quantized Diffusion Model Based Speech Bandwidth Extensioncited by 4
  32. 2024G2D2: Gradient-Guided Discrete Diffusion for Inverse Problem Solvingcited by 4
  33. 2024BIFR"OST: 3D-Aware Image compositing with Language Instructionscited by 4
  34. 2024HypDAE: Hyperbolic Diffusion Autoencoders for Hierarchical Few-shot Image Generationcited by 4
  35. 2025Purrception: Variational Flow Matching for Vector-Quantized Image Generationcited by 4
  36. 2024JPEG-LM: LLMs as Image Generators with Canonical Codec Representationscited by 3
  37. 2024Unlocking the Capabilities of Masked Generative Models for Image Synthesis via Self-Guidancecited by 3
  38. 2024PersonaMagic: Stage-Regulated High-Fidelity Face Customization with Tandem Equilibriumcited by 3

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Accurate Compression of Text-to-Image Diffusion Models via Vector Quantization
  2. 2024PQD: Post-training Quantization for Efficient Diffusion Models
  3. 2021Global Context with Discrete Diffusion in Vector Quantised Modelling for Image Generation
  4. 2025AQUATIC-Diff: Additive Quantization for Truly Tiny Compressed Diffusion Models
  5. 2024VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers
  6. 2025Diffusion Model Quantization: A Review
  7. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models
  8. 2024QNCD: Quantization Noise Correction for Diffusion Models