mlsanityData to 14 Oct 2026

DiVAE: Photorealistic Images Synthesis with Denoising Diffusion Decoder

arXiv 2206.00386 ↗First posted 1 Jun 2022cs.CV cs.AIcited by 27

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Recently most successful image synthesis models are multi stage process to combine the advantages of different methods, which always includes a VAE-like model for faithfully reconstructing embedding to image and a prior model to generate image embedding. At the same time, diffusion models have shown be capacity to generate high-quality synthetic images. Our work proposes a VQ-VAE architecture model with a diffusion decoder (DiVAE) to work as the reconstructing component in image synthesis. We explore how to input image embedding into diffusion model for excellent performance and find that simple modification on diffusion's UNet can achieve it. Training on ImageNet, Our model achieves state-of-the-art results and generates more photorealistic images specifically. In addition, we apply the DiVAE with an Auto-regressive generator on conditional synthesis tasks to perform more human-feeling and detailed samples.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 40 of its 49 references that are in the map. First posted on arXiv 1 Jun 2022.

Builds on

40 of 49 references.

Closest ancestors · 8

Same direction, newest first

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  2. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  4. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  5. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  6. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  7. 2020Denoising Diffusion Implicit Models6 Oct
  8. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Generative Adversarial Networkscited by 5,627
  7. 2017Neural Discrete Representation Learningcited by 4,712
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2013Auto-Encoding Variational Bayescited by 4,053
  10. 2018Neural Ordinary Differential Equationscited by 3,779
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
20 more
  1. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  2. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  3. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  4. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  5. 2016Pixel Recurrent Neural Networkscited by 816
  6. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  7. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  8. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  9. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  10. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 417
  11. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  12. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 262
  13. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  14. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 256
  15. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  16. 2019Diagnosing and Enhancing VAE Modelscited by 193
  17. 2019Controllable Text-to-Image Generationcited by 187
  18. 2021M6: A Chinese Multimodal Pretrainercited by 91
  19. 2019Dual Adversarial Inference for Text-to-Image Synthesiscited by 17
  20. 2018Adversarial Learning of Semantic Relevance in Text to Image Synthesiscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 7; 2024: 7; 2025: 6; 2026: 5.

27 papers.

Sort:

By area:

  1. 2022Diffusion Models in Vision: A Surveycited by 1,015
  2. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  3. 20234M: Massively Multimodal Masked Modelingcited by 120
  4. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 94
  5. 2024Scaling the Codebook Size of VQGAN to 100,000 with a Utilization Rate of 99%cited by 68
  6. 20244M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalitiescited by 52
  7. 2022Efficient Diffusion Models for Vision: A Surveycited by 51
  8. 2023WaveDM: Wavelet-Based Diffusion Models for Image Restorationcited by 47
  9. 2023IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AIcited by 35
  10. 2023HD-Fusion: Detailed Text-to-3D Generation Leveraging Multiple Noise Estimationcited by 27
  11. 2026Unified Latents (UL): How to train your latentscited by 20
  12. 2024Epsilon-VAE: Denoising as Visual Decodingcited by 16
15 more
  1. 2023Personalized Face Inpainting with Diffusion Models by Parallel Visual Attentioncited by 13
  2. 2023Null-text Guidance in Diffusion Models is Secretly a Cartoon-style Creatorcited by 12
  3. 2024XQ-GAN: An Open-source Image Tokenization Framework for Autoregressive Generationcited by 12
  4. 2025Image Tokenizer Needs Post-Trainingcited by 12
  5. 2025Robust Latent Matters: Boosting Image Generation with Sampling Error Synthesiscited by 10
  6. 2025CODA: Repurposing Continuous VAEs for Discrete Tokenizationcited by 8
  7. 2025SSDD: Single-Step Diffusion Decoder for Efficient Image Tokenizationcited by 7
  8. 2026PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusioncited by 7
  9. 2024Sample what you cant compresscited by 5
  10. 2025Layton: Latent Consistency Tokenizer for 1024-pixel Image Reconstruction and Generation by 256 Tokenscited by 3
  11. 2025DGAE: Diffusion-Guided Autoencoder for Efficient Latent Representation Learningcited by 2
  12. 2026Probabilistic Precipitation Nowcasting with Rectified Flow Transformerscited by 2
  13. 2026Distributional Matching for Vector Quantization: A Unified Theoretical and Empirical Frameworkcited by 1
  14. 2026VQ-Transplant: Efficient VQ-Module Integration for Pre-trained Visual Tokenizerscited by 1
  15. 2023A Diffusion Model Based Quality Enhancement Method for HEVC Compressed Videocited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents
  2. 2024Conditional Image Synthesis with Diffusion Models: A Survey
  3. 2022Image Generation with Multimodal Priors using Denoising Diffusion Probabilistic Models
  4. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
  5. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance
  6. 2024Comparative Analysis of Generative Models: Enhancing Image Synthesis with VAEs, GANs, and Stable Diffusion
  7. 2024Ultra-High-Resolution Image Synthesis with Pyramid Diffusion Model
  8. 2025Diffusion Restoration Adapter for Real-World Image Restoration