# Instance-aware Image Colorization

arXiv: https://arxiv.org/abs/2005.10825 · HTML: https://mlsanity.com/paper/2005.10825/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2020.

## Built on by (most cited first, 30 of 76)

- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Towards Vivid and Diverse Image Colorization with Generative Color Prior](https://mlsanity.com/paper/2108.08826/) (2021)
- [Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2108.04547/) (2021)
- [DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders](https://mlsanity.com/paper/2212.11613/) (2022)
- [BigColor: Colorization using a Generative Color Prior for Natural Images](https://mlsanity.com/paper/2207.09685/) (2022)
- [Control Color: Multimodal Diffusion-based Interactive Image Colorization](https://mlsanity.com/paper/2402.10855/) (2024)
- [Diffusing Colors: Image Colorization with Text Guided Diffusion](https://mlsanity.com/paper/2312.04145/) (2023)
- [Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning](https://mlsanity.com/paper/2110.04562/) (2021)
- [GLEAN: Generative Latent Bank for Image Super-Resolution and Beyond](https://mlsanity.com/paper/2207.14812/) (2022)
- [PalGAN: Image Colorization with Palette Generative Adversarial Networks](https://mlsanity.com/paper/2210.11204/) (2022)
- [Improved Diffusion-based Image Colorization via Piggybacked Models](https://mlsanity.com/paper/2304.11105/) (2023)
- [BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization](https://mlsanity.com/paper/2212.02268/) (2022)
- [VCGAN: Video Colorization with Hybrid Generative Adversarial Network](https://mlsanity.com/paper/2104.12357/) (2021)
- [Deep Edge-Aware Interactive Colorization against Color-Bleeding Effects](https://mlsanity.com/paper/2107.01619/) (2021)
- [Contrastive Spatio-Temporal Pretext Learning for Self-supervised Video Representation](https://mlsanity.com/paper/2112.08913/) (2021)
- [Video Colorization with Pre-trained Text-to-Image Diffusion Models](https://mlsanity.com/paper/2306.01732/) (2023)
- [Follow-Your-Color: Multi-Instance Sketch Colorization](https://mlsanity.com/paper/2503.16948/) (2025)
- [Color2Embed: Fast Exemplar-Based Image Colorization using Color Embeddings](https://mlsanity.com/paper/2106.08017/) (2021)
- [SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution](https://mlsanity.com/paper/2201.04364/) (2022)
- [Pik-Fix: Restoring and Colorizing Old Photos](https://mlsanity.com/paper/2205.01902/) (2022)
- [ColorMNet: A Memory-based Deep Spatial-Temporal Feature Propagation Network for Video Colorization](https://mlsanity.com/paper/2404.06251/) (2024)
- [UniColor: A Unified Framework for Multi-Modal Colorization with Transformer](https://mlsanity.com/paper/2209.11223/) (2022)
- [Colorizing Monochromatic Radiance Fields](https://mlsanity.com/paper/2402.12184/) (2024)
- [Automatic Controllable Colorization via Imagination](https://mlsanity.com/paper/2404.05661/) (2024)
- [Focusing on Persons: Colorizing Old Images Learning from Modern Historical Movies](https://mlsanity.com/paper/2108.06515/) (2021)
- [Efficient Anomaly Detection Using Self-Supervised Multi-Cue Tasks](https://mlsanity.com/paper/2111.12379/) (2021)
- [Cooperative Colorization: Exploring Latent Cross-Domain Priors for NIR Image Spectrum Translation](https://mlsanity.com/paper/2308.03348/) (2023)
- [MultiColor: Image Colorization by Learning from Multiple Color Spaces](https://mlsanity.com/paper/2408.04172/) (2024)
- [L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors](https://mlsanity.com/paper/2305.15217/) (2023)
- [ChromaDistill: Colorizing Monochrome Radiance Fields with Knowledge Distillation](https://mlsanity.com/paper/2309.07668/) (2023)

## Abstract (arXiv)

Image colorization is inherently an ill-posed problem with multi-modal uncertainty. Previous methods leverage the deep neural network to map input grayscale images to plausible color outputs directly. Although these learning-based methods have shown impressive performance, they usually fail on the input images that contain multiple objects. The leading cause is that existing models perform learning and colorization on the entire image. In the absence of a clear figure-ground separation, these models cannot effectively locate and learn meaningful object-level semantics. In this paper, we propose a method for achieving instance-aware colorization. Our network architecture leverages an off-the-shelf object detector to obtain cropped object images and uses an instance colorization network to extract object-level features. We use a similar network to extract the full-image features and apply a fusion module to full object-level and image-level features to predict the final colors. Both colorization networks and fusion modules are learned from a large-scale dataset. Experimental results show that our work outperforms existing methods on different quality metrics and achieves state-of-the-art performance on image colorization.