# MultiColor: Image Colorization by Learning from Multiple Color Spaces

arXiv: https://arxiv.org/abs/2408.04172 · HTML: https://mlsanity.com/paper/2408.04172/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 8 Aug 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Underwater Image Enhancement via Medium Transmission-Guided Multi-Color Space Embedding](https://mlsanity.com/paper/2104.13015/) (2021)
- [MAST: A Memory-Augmented Self-supervised Tracker](https://mlsanity.com/paper/2002.07793/) (2020)
- [Bringing Old Photos Back to Life](https://mlsanity.com/paper/2004.09484/) (2020)
- [Colorization Transformer](https://mlsanity.com/paper/2102.04432/) (2021)
- [Deep Exemplar-based Colorization](https://mlsanity.com/paper/1807.06587/) (2018)
- [Instance-aware Image Colorization](https://mlsanity.com/paper/2005.10825/) (2020)
- [Deep Colorization](https://mlsanity.com/paper/1605.00075/) (2016)
- [ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution](https://mlsanity.com/paper/1907.09837/) (2019)
- [Towards Vivid and Diverse Image Colorization with Generative Color Prior](https://mlsanity.com/paper/2108.08826/) (2021)
- [DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders](https://mlsanity.com/paper/2212.11613/) (2022)
- [BigColor: Colorization using a Generative Color Prior for Natural Images](https://mlsanity.com/paper/2207.09685/) (2022)
- [ColorNet: Investigating the importance of color spaces for image classification](https://mlsanity.com/paper/1902.00267/) (2019)
- [Bringing Old Films Back to Life](https://mlsanity.com/paper/2203.17276/) (2022)
- [Structural Consistency and Controllability for Diverse Colorization](https://mlsanity.com/paper/1809.02129/) (2018)
- [Progressive Scene Text Erasing with Self-Supervision](https://mlsanity.com/paper/2207.11469/) (2022)

## Built on by (most cited first, 6 of 6)

- [Consistent Video Colorization via Palette Guidance](https://mlsanity.com/paper/2501.19331/) (2025)
- [Language-based Image Colorization: A Benchmark and Beyond](https://mlsanity.com/paper/2503.14974/) (2025)
- [FusionNet: Multi-model Linear Fusion Framework for Low-light Image Enhancement](https://mlsanity.com/paper/2504.19295/) (2025)
- [End-to-End Fine-Tuning of 3D Texture Generation using Differentiable Rewards](https://mlsanity.com/paper/2506.18331/) (2025)
- [Beyond Fixed Luminance: Towards Panchromatic and Orthochromatic Image Colorization](https://mlsanity.com/paper/2608.10798/) (2026)
- [MCIQA-2K: A Multi-Dimensional Dataset and No-Reference Quality Assessment Benchmark for Colorized Images](https://mlsanity.com/paper/2609.14495/) (2026)

## Abstract (arXiv)

Deep networks have shown impressive performance in the image restoration tasks, such as image colorization. However, we find that previous approaches rely on the digital representation from single color model with a specific mapping function, a.k.a., color space, during the colorization pipeline. In this paper, we first investigate the modeling of different color spaces, and find each of them exhibiting distinctive characteristics with unique distribution of colors. The complementarity among multiple color spaces leads to benefits for the image colorization task. We present MultiColor, a new learning-based approach to automatically colorize grayscale images that combines clues from multiple color spaces. Specifically, we employ a set of dedicated colorization modules for individual color space. Within each module, a transformer decoder is first employed to refine color query embeddings and then a color mapper produces color channel prediction using the embeddings and semantic features. With these predicted color channels representing various color spaces, a complementary network is designed to exploit the complementarity and generate pleasing and reasonable colorized images. We conduct extensive experiments on real-world datasets, and the results demonstrate superior performance over the state-of-the-arts.