mlsanityData to 7 Oct 2026

MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generation

arXiv 2209.09002 ↗First posted 19 Sep 2022cs.CVcited by 125

Not on the map. Why?

Cited by 25 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Although two-stage Vector Quantized (VQ) generative models allow for synthesizing high-fidelity and high-resolution images, their quantization operator encodes similar patches within an image into the same index, resulting in a repeated artifact for similar adjacent regions using existing decoder architectures. To address this issue, we propose to incorporate the spatially conditional normalization to modulate the quantized vectors so as to insert spatially variant information to the embedded index maps, encouraging the decoder to generate more photorealistic images. Moreover, we use multichannel quantization to increase the recombination capability of the discrete codes without increasing the cost of model and codebook. Additionally, to generate discrete tokens at the second stage, we adopt a Masked Generative Image Transformer (MaskGIT) to learn an underlying prior distribution in the compressed latent space, which is much faster than the conventional autoregressive model. Experiments on two benchmark datasets demonstrate that our proposed modulated VQGAN is able to greatly improve the reconstructed image quality as well as provide high-fidelity image generation.

Key facts. Cited by 125 papers in the map (data through 7 Oct 2026). Builds on 43 of its 51 references that are in the map. First posted on arXiv 19 Sep 2022.

Builds on

43 of 51 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
31 more
  1. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  2. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  3. 2016Improved Techniques for Training GANscited by 2,938
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2017Improved Training of Wasserstein GANscited by 2,223
  6. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  7. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  8. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  9. 2018Group Normalizationcited by 1,514
  10. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  11. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  12. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  13. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  14. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  15. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  16. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  17. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  18. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  19. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  20. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  21. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  22. 2017Convolutional Sequence to Sequence Learningcited by 550
  23. 2016A Learned Representation For Artistic Stylecited by 340
  24. 2021Generating Images with Sparse Representationscited by 262
  25. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  26. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  27. 2020Latent Video Transformercited by 106
  28. 2020Navigating the GAN Parameter Space for Semantic Image Editingcited by 49
  29. 2021Global Context with Discrete Diffusion in Vector Quantised Modelling for Image Generationcited by 40
  30. 2021Improving GAN Equilibrium by Raising Spatial Awarenesscited by 22
  31. 2022High-Quality Pluralistic Image Completion via Code Shared VQGANcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 15; 2024: 30; 2025: 54; 2026: 24.

125 papers.

Sort:

By area:

  1. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  2. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  3. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  4. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  5. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
  6. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  7. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  8. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 258
  9. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
  10. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
  11. 2025Unified Vision-Language-Action Modelcited by 128
  12. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 127
38 more
  1. 2023Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusioncited by 114
  2. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 111
  3. 2024MaskBit: Embedding-free Image Generation via Bit Tokenscited by 96
  4. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  5. 2023Free3D: Consistent Novel View Synthesis without 3D Representationcited by 72
  6. 2024Scaling the Codebook Size of VQGAN to 100,000 with a Utilization Rate of 99%cited by 68
  7. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPTcited by 66
  8. 2023Towards Accurate Image Coding: Improved Autoregressive Image Generation with Dynamic Vector Quantizationcited by 59
  9. 2023Online Clustered Codebookcited by 55
  10. 2025NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scalecited by 55
  11. 2025ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinementcited by 53
  12. 2025UniToken: Harmonizing Multimodal Understanding and Generation through Unified Visual Encodingcited by 49
  13. 2024LARP: Tokenizing Videos with a Learned Autoregressive Generative Priorcited by 46
  14. 2025DualToken: Towards Unifying Visual Understanding and Generation with Dual Visual Vocabulariescited by 42
  15. 2022Unified Discrete Diffusion for Simultaneous Vision-Language Generationcited by 39
  16. 2024Autoregressive Models in Vision: A Surveycited by 39
  17. 2025EVEv2: Improved Baselines for Encoder-Free Vision-Language Modelscited by 38
  18. 2025Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Processcited by 37
  19. 2025FastVAR: Linear Visual Autoregressive Modeling via Cached Token Pruningcited by 34
  20. 2022Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesiscited by 31
  21. 2025AToken: A Unified Tokenizer for Visioncited by 30
  22. 2023Not All Image Regions Matter: Masked Vector Quantization for Autoregressive Image Generationcited by 28
  23. 2024Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamicscited by 28
  24. 2024Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Surveycited by 27
  25. 2025Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokenscited by 27
  26. 2024How to Trace Latent Generative Model Generated Images without Artificial Watermark?cited by 25
  27. 2025UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generationcited by 23
  28. 2023Vector Quantized Wasserstein Auto-Encodercited by 22
  29. 2025FlexVAR: Flexible Visual Autoregressive Modeling without Residual Predictioncited by 22
  30. 2023LaMD: Latent Motion Diffusion for Image-Conditional Video Generationcited by 21
  31. 2023Diffusion Model with Perceptual Losscited by 21
  32. 2024TCSinger: Zero-Shot Singing Voice Synthesis with Style Transfer and Multi-Level Style Controlcited by 20
  33. 2025Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Image Generationcited by 20
  34. 2025VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoocited by 20
  35. 2025Hybrid Token Compression for Vision-Language Modelscited by 19
  36. 2023Kandinsky 3.0 Technical Reportcited by 18
  37. 2025CAT: Content-Adaptive Image Tokenizationcited by 18
  38. 2025DetailFlow: 1D Coarse-to-Fine Autoregressive Image Generation via Next-Detail Predictioncited by 17

50 most cited of 125 All 125 (JSON)