# Dequantization and Color Transfer with Diffusion Models

arXiv: https://arxiv.org/abs/2307.02698 · HTML: https://mlsanity.com/paper/2307.02698/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 6 Jul 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 30)

- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model](https://mlsanity.com/paper/2212.05034/) (2022)
- [StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis](https://mlsanity.com/paper/2301.09515/) (2023)
- [Deep Photo Style Transfer](https://mlsanity.com/paper/1703.07511/) (2017)
- [Real-Time User-Guided Image Colorization with Learned Deep Priors](https://mlsanity.com/paper/1705.02999/) (2017)
- [Exploring the structure of a real-time, arbitrary neural artistic stylization network](https://mlsanity.com/paper/1705.06830/) (2017)
- [Image inpainting: A review](https://mlsanity.com/paper/1909.06399/) (2019)
- [Towards Flexible Blind JPEG Artifacts Removal](https://mlsanity.com/paper/2109.14573/) (2021)
- [Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses](https://mlsanity.com/paper/1701.08893/) (2017)
- [Zero-Shot Contrastive Loss for Text-Guided Diffusion Image Style Transfer](https://mlsanity.com/paper/2303.08622/) (2023)
- [An Overview of Color Transfer and Style Transfer for Images and Videos](https://mlsanity.com/paper/2204.13339/) (2022)
- [Controlled GAN-Based Creature Synthesis via a Challenging Game Art Dataset -- Addressing the Noise-Latent Trade-Off](https://mlsanity.com/paper/2108.08922/) (2021)
- [Image De-Quantization Using Generative Models as Priors](https://mlsanity.com/paper/2007.07923/) (2020)

## Built on by (most cited first, 4 of 4)

- [Blocks2World: Controlling Realistic Scenes with Editable Primitives](https://mlsanity.com/paper/2307.03847/) (2023)
- [Denoising Monte Carlo Renders with Diffusion Models](https://mlsanity.com/paper/2404.00491/) (2024)
- [Generative Blocks World: Moving Things Around in Pictures](https://mlsanity.com/paper/2506.20703/) (2025)
- [The Latent Color Subspace: Emergent Order in High-Dimensional Chaos](https://mlsanity.com/paper/2603.12261/) (2026)

## Abstract (arXiv)

We demonstrate an image dequantizing diffusion model that enables novel edits on natural images. We propose operating on quantized images because they offer easy abstraction for patch-based edits and palette transfer. In particular, we show that color palettes can make the output of the diffusion model easier to control and interpret. We first establish that existing image restoration methods are not sufficient, such as JPEG noise reduction models. We then demonstrate that our model can generate natural images that respect the color palette the user asked for. For palette transfer, we propose a method based on weighted bipartite matching. We then show that our model generates plausible images even after extreme palette transfers, respecting user query. Our method can optionally condition on the source texture in part or all of the image. In doing so, we overcome a common problem in existing image colorization methods that are unable to produce colors with a different luminance than the input. We evaluate several possibilities for texture conditioning and their trade-offs, including luminance, image gradients, and thresholded gradients, the latter of which performed best in maintaining texture and color control simultaneously. Our method can be usefully extended to another practical edit: recoloring patches of an image while respecting the source texture. Our procedure is supported by several qualitative and quantitative evaluations.