# Learning Diverse Image Colorization

arXiv: https://arxiv.org/abs/1612.01958 · HTML: https://mlsanity.com/paper/1612.01958/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2016.

## Built on by (most cited first, 30 of 43)

- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Colorization Transformer](https://mlsanity.com/paper/2102.04432/) (2021)
- [OBJECT 3DIT: Language-guided 3D-aware Image Editing](https://mlsanity.com/paper/2307.11073/) (2023)
- [Towards Vivid and Diverse Image Colorization with Generative Color Prior](https://mlsanity.com/paper/2108.08826/) (2021)
- [Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks](https://mlsanity.com/paper/2112.01715/) (2021)
- [DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders](https://mlsanity.com/paper/2212.11613/) (2022)
- [Conditional Invertible Neural Networks for Diverse Image-to-Image Translation](https://mlsanity.com/paper/2105.02104/) (2021)
- [Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning](https://mlsanity.com/paper/2110.04562/) (2021)
- [Invertible Rescaling Network and Its Extensions](https://mlsanity.com/paper/2210.04188/) (2022)
- [PalGAN: Image Colorization with Palette Generative Adversarial Networks](https://mlsanity.com/paper/2210.11204/) (2022)
- [Generative Models in Decision Making: A Survey](https://mlsanity.com/paper/2502.17100/) (2025)
- [VCGAN: Video Colorization with Hybrid Generative Adversarial Network](https://mlsanity.com/paper/2104.12357/) (2021)
- [Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization](https://mlsanity.com/paper/2112.01335/) (2021)
- [Wavelet Transform-assisted Adaptive Generative Modeling for Colorization](https://mlsanity.com/paper/2107.04261/) (2021)
- [SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution](https://mlsanity.com/paper/2201.04364/) (2022)
- [UniColor: A Unified Framework for Multi-Modal Colorization with Transformer](https://mlsanity.com/paper/2209.11223/) (2022)
- [Backdoor Attacks against Image-to-Image Networks](https://mlsanity.com/paper/2407.10445/) (2024)
- [L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors](https://mlsanity.com/paper/2305.15217/) (2023)
- [ChromaDistill: Colorizing Monochrome Radiance Fields with Knowledge Distillation](https://mlsanity.com/paper/2309.07668/) (2023)
- [StyLitGAN: Prompting StyleGAN to Produce New Illumination Conditions](https://mlsanity.com/paper/2205.10351/) (2022)
- [CHIMLE: Conditional Hierarchical IMLE for Multimodal Conditional Image Synthesis](https://mlsanity.com/paper/2211.14286/) (2022)
- [StyleFlow For Content-Fixed Image to Image Translation](https://mlsanity.com/paper/2207.01909/) (2022)
- [An Overview on the Generation and Detection of Synthetic and Manipulated Satellite Images](https://mlsanity.com/paper/2209.08984/) (2022)
- [Two Decades of Colorization and Decolorization for Images and Videos](https://mlsanity.com/paper/2204.13322/) (2022)
- [Adaptive Selection of Sampling-Reconstruction in Fourier Compressed Sensing](https://mlsanity.com/paper/2409.11738/) (2024)
- [TUCaN: Progressively Teaching Colourisation to Capsules](https://mlsanity.com/paper/2106.15176/) (2021)
- [Bias in Automated Image Colorization: Metrics and Error Types](https://mlsanity.com/paper/2202.08143/) (2022)
- [Analysis of Different Losses for Deep Learning Image Colorization](https://mlsanity.com/paper/2204.02980/) (2022)
- [A Benchmarking Protocol for SAR Colorization: From Regression to Deep Learning Approaches](https://mlsanity.com/paper/2310.08705/) (2023)
- [ColorVein: Colorful Cancelable Vein Biometrics](https://mlsanity.com/paper/2504.14253/) (2025)

## Abstract (arXiv)

Colorization is an ambiguous problem, with multiple viable colorizations for a single grey-level image. However, previous methods only produce the single most probable colorization. Our goal is to model the diversity intrinsic to the problem of colorization and produce multiple colorizations that display long-scale spatial co-ordination. We learn a low dimensional embedding of color fields using a variational autoencoder (VAE). We construct loss terms for the VAE decoder that avoid blurry outputs and take into account the uneven distribution of pixel colors. Finally, we build a conditional model for the multi-modal distribution between grey-level image and the color field embeddings. Samples from this conditional model result in diverse colorization. We demonstrate that our method obtains better diverse colorizations than a standard conditional variational autoencoder (CVAE) model, as well as a recently proposed conditional generative adversarial network (cGAN).