# Structural Consistency and Controllability for Diverse Colorization

arXiv: https://arxiv.org/abs/1809.02129 · HTML: https://mlsanity.com/paper/1809.02129/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2018.

## Built on by (most cited first, 15 of 15)

- [Colorization Transformer](https://mlsanity.com/paper/2102.04432/) (2021)
- [Towards Vivid and Diverse Image Colorization with Generative Color Prior](https://mlsanity.com/paper/2108.08826/) (2021)
- [DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders](https://mlsanity.com/paper/2212.11613/) (2022)
- [S2AC: Energy-Based Reinforcement Learning with Stein Soft Actor Critic](https://mlsanity.com/paper/2405.00987/) (2024)
- [SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution](https://mlsanity.com/paper/2201.04364/) (2022)
- [UniColor: A Unified Framework for Multi-Modal Colorization with Transformer](https://mlsanity.com/paper/2209.11223/) (2022)
- [Colorizing Monochromatic Radiance Fields](https://mlsanity.com/paper/2402.12184/) (2024)
- [Winner-takes-all learners are geometry-aware conditional density estimators](https://mlsanity.com/paper/2406.04706/) (2024)
- [MultiColor: Image Colorization by Learning from Multiple Color Spaces](https://mlsanity.com/paper/2408.04172/) (2024)
- [POET: Supporting Prompting Creativity and Personalization with Automated Expansion of Text-to-Image Generation](https://mlsanity.com/paper/2504.13392/) (2025)
- [ChromaDistill: Colorizing Monochrome Radiance Fields with Knowledge Distillation](https://mlsanity.com/paper/2309.07668/) (2023)
- [Towards Photorealistic Colorization by Imagination](https://mlsanity.com/paper/2108.09195/) (2021)
- [ParaColorizer: Realistic Image Colorization using Parallel Generative Networks](https://mlsanity.com/paper/2208.08295/) (2022)
- [Computer-aided Colorization State-of-the-science: A Survey](https://mlsanity.com/paper/2410.02288/) (2024)
- [Conditional Normalizing Flows for Forward and Backward Joint State and Parameter Estimation](https://mlsanity.com/paper/2601.07013/) (2026)

## Abstract (arXiv)

Colorizing a given gray-level image is an important task in the media and advertising industry. Due to the ambiguity inherent to colorization (many shades are often plausible), recent approaches started to explicitly model diversity. However, one of the most obvious artifacts, structural inconsistency, is rarely considered by existing methods which predict chrominance independently for every pixel. To address this issue, we develop a conditional random field based variational auto-encoder formulation which is able to achieve diversity while taking into account structural consistency. Moreover, we introduce a controllability mecha- nism that can incorporate external constraints from diverse sources in- cluding a user interface. Compared to existing baselines, we demonstrate that our method obtains more diverse and globally consistent coloriza- tions on the LFW, LSUN-Church and ILSVRC-2015 datasets.