# Vector Quantized Image-to-Image Translation

arXiv: https://arxiv.org/abs/2207.13286 · HTML: https://mlsanity.com/paper/2207.13286/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 27 Jul 2022.

## Builds on (most cited first, 30 of 30)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1808.00948/) (2018)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation](https://mlsanity.com/paper/1907.10830/) (2019)
- [Image to Image Translation for Domain Adaptation](https://mlsanity.com/paper/1712.00479/) (2017)
- [Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification](https://mlsanity.com/paper/1711.07027/) (2017)
- [Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis](https://mlsanity.com/paper/1903.05628/) (2019)
- [Boundless: Generative Adversarial Networks for Image Extension](https://mlsanity.com/paper/1908.07007/) (2019)
- [InfinityGAN: Towards Infinite-Pixel Image Synthesis](https://mlsanity.com/paper/2104.03963/) (2021)
- [In&Out : Diverse Image Outpainting via GAN Inversion](https://mlsanity.com/paper/2104.00675/) (2021)
- [Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning](https://mlsanity.com/paper/2203.02573/) (2022)
- [RetrieveGAN: Image Synthesis via Differentiable Patch Retrieval](https://mlsanity.com/paper/2007.08513/) (2020)
- [Semantic View Synthesis](https://mlsanity.com/paper/2008.10598/) (2020)
- [Controllable Image Synthesis via SegVAE](https://mlsanity.com/paper/2007.08397/) (2020)
- [Continuous and Diverse Image-to-Image Translation via Signed Attribute Vectors](https://mlsanity.com/paper/2011.01215/) (2020)

## Built on by (most cited first, 16 of 16)

- [DiffCollage: Parallel Generation of Large Content with Diffusion Models](https://mlsanity.com/paper/2303.17076/) (2023)
- [Exploiting Diffusion Prior for Generalizable Dense Prediction](https://mlsanity.com/paper/2311.18832/) (2023)
- [MaskSketch: Unpaired Structure-guided Masked Image Generation](https://mlsanity.com/paper/2302.05496/) (2023)
- [StainDiffuser: MultiTask Dual Diffusion Model for Virtual Staining](https://mlsanity.com/paper/2403.11340/) (2024)
- [Unaligned 2D to 3D Translation with Conditional Vector-Quantized Code Diffusion using Transformers](https://mlsanity.com/paper/2308.14152/) (2023)
- [One-for-All: Towards Universal Domain Translation with a Single StyleGAN](https://mlsanity.com/paper/2310.14222/) (2023)
- [MuTri: Multi-view Tri-alignment for OCT to OCTA 3D Image Translation](https://mlsanity.com/paper/2504.01428/) (2025)
- [A Training-Free Style-aligned Image Generation with Scale-wise Autoregressive Model](https://mlsanity.com/paper/2504.06144/) (2025)
- [Unsupervised Variational Translator for Bridging Image Restoration and High-Level Vision Tasks](https://mlsanity.com/paper/2408.08149/) (2024)
- [Image-to-Image Translation Framework Embedded with Rotation Symmetry Priors](https://mlsanity.com/paper/2604.12805/) (2026)
- [Scene Depth Estimation from Traditional Oriental Landscape Paintings](https://mlsanity.com/paper/2403.03408/) (2024)
- [The Interpretability of Codebooks in Model-Based Reinforcement Learning is Limited](https://mlsanity.com/paper/2407.19532/) (2024)
- [IMPLICITSTAINER: Resolution Agnostic Data-Efficient Virtual Staining Using Neural Implicit Functions](https://mlsanity.com/paper/2505.09831/) (2025)
- [Image Translation with Kernel Prediction Networks for Semantic Segmentation](https://mlsanity.com/paper/2507.08554/) (2025)
- [Building Trust in Virtual Immunohistochemistry: Automated Assessment of Image Quality](https://mlsanity.com/paper/2511.04615/) (2025)
- [PAINT: Pathology-Aware Integrated Next-Scale Transformation for Virtual Immunohistochemistry](https://mlsanity.com/paper/2601.16024/) (2026)

## Abstract (arXiv)

Current image-to-image translation methods formulate the task with conditional generation models, leading to learning only the recolorization or regional changes as being constrained by the rich structural information provided by the conditional contexts. In this work, we propose introducing the vector quantization technique into the image-to-image translation framework. The vector quantized content representation can facilitate not only the translation, but also the unconditional distribution shared among different domains. Meanwhile, along with the disentangled style representation, the proposed method further enables the capability of image extension with flexibility in both intra- and inter-domains. Qualitative and quantitative experiments demonstrate that our framework achieves comparable performance to the state-of-the-art image-to-image translation and image extension methods. Compared to methods for individual tasks, the proposed method, as a unified framework, unleashes applications combining image-to-image translation, unconditional generation, and image extension altogether. For example, it provides style variability for image generation and extension, and equips image-to-image translation with further extension capabilities.