# Pixel-level Semantics Guided Image Colorization

arXiv: https://arxiv.org/abs/1808.01597 · HTML: https://mlsanity.com/paper/1808.01597/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Aug 2018.

## Built on by (most cited first, 15 of 15)

- [Towards Vivid and Diverse Image Colorization with Generative Color Prior](https://mlsanity.com/paper/2108.08826/) (2021)
- [DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders](https://mlsanity.com/paper/2212.11613/) (2022)
- [Deep Edge-Aware Interactive Colorization against Color-Bleeding Effects](https://mlsanity.com/paper/2107.01619/) (2021)
- [Color2Embed: Fast Exemplar-Based Image Colorization using Color Embeddings](https://mlsanity.com/paper/2106.08017/) (2021)
- [Pik-Fix: Restoring and Colorizing Old Photos](https://mlsanity.com/paper/2205.01902/) (2022)
- [L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors](https://mlsanity.com/paper/2305.15217/) (2023)
- [DiffColor: Toward High Fidelity Text-Guided Image Colorization with Diffusion Models](https://mlsanity.com/paper/2308.01655/) (2023)
- [TUCaN: Progressively Teaching Colourisation to Capsules](https://mlsanity.com/paper/2106.15176/) (2021)
- [Learning Spatially Decoupled Color Representations for Facial Image Colorization](https://mlsanity.com/paper/2412.07203/) (2024)
- [Collaboration among Image and Object Level Features for Image Colourisation](https://mlsanity.com/paper/2101.07576/) (2021)
- [ROMNet: Renovate the Old Memories](https://mlsanity.com/paper/2202.02606/) (2022)
- [Human vs Objective Evaluation of Colourisation Performance](https://mlsanity.com/paper/2204.05200/) (2022)
- [ParaColorizer: Realistic Image Colorization using Parallel Generative Networks](https://mlsanity.com/paper/2208.08295/) (2022)
- [UW-ProCCaps: UnderWater Progressive Colourisation with Capsules](https://mlsanity.com/paper/2307.01091/) (2023)
- [Computer-aided Colorization State-of-the-science: A Survey](https://mlsanity.com/paper/2410.02288/) (2024)

## Abstract (arXiv)

While many image colorization algorithms have recently shown the capability of producing plausible color versions from gray-scale photographs, they still suffer from the problems of context confusion and edge color bleeding. To address context confusion, we propose to incorporate the pixel-level object semantics to guide the image colorization. The rationale is that human beings perceive and distinguish colors based on the object's semantic categories. We propose a hierarchical neural network with two branches. One branch learns what the object is while the other branch learns the object's colors. The network jointly optimizes a semantic segmentation loss and a colorization loss. To attack edge color bleeding we generate more continuous color maps with sharp edges by adopting a joint bilateral upsamping layer at inference. Our network is trained on PASCAL VOC2012 and COCO-stuff with semantic segmentation labels and it produces more realistic and finer results compared to the colorization state-of-the-art.