# Guiding Users to Where to Give Color Hints for Efficient Interactive Sketch Colorization via Unsupervised Region Prioritization

arXiv: https://arxiv.org/abs/2210.14270 · HTML: https://mlsanity.com/paper/2210.14270/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 44 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [Segmentation from Natural Language Expressions](https://mlsanity.com/paper/1603.06180/) (2016)
- [Real-Time User-Guided Image Colorization with Learned Deep Priors](https://mlsanity.com/paper/1705.02999/) (2017)
- [SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color](https://mlsanity.com/paper/1902.06838/) (2019)
- [Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language](https://mlsanity.com/paper/1810.11919/) (2018)
- [Scribbler: Controlling Deep Image Synthesis with Sketch and Color](https://mlsanity.com/paper/1612.00835/) (2016)
- [Instance-aware Image Colorization](https://mlsanity.com/paper/2005.10825/) (2020)
- [Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence](https://mlsanity.com/paper/2005.05207/) (2020)
- [Going to Extremes: Weakly Supervised Medical Image Segmentation](https://mlsanity.com/paper/2009.11988/) (2020)
- [Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss](https://mlsanity.com/paper/1908.05840/) (2019)
- [Bipartite Graph Reasoning GANs for Person Image Generation](https://mlsanity.com/paper/2008.04381/) (2020)
- [Sequential Attention GAN for Interactive Image Editing](https://mlsanity.com/paper/1812.08352/) (2018)
- [User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks](https://mlsanity.com/paper/1808.03240/) (2018)
- [Coloring With Limited Data: Few-Shot Colorization via Memory-Augmented Networks](https://mlsanity.com/paper/1906.11888/) (2019)
- [Unsupervised Diverse Colorization via Generative Adversarial Networks](https://mlsanity.com/paper/1702.06674/) (2017)
- [FaceShop: Deep Sketch-based Face Image Editing](https://mlsanity.com/paper/1804.08972/) (2018)
- [MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing](https://mlsanity.com/paper/2010.16417/) (2020)
- [Constrained Generative Adversarial Networks for Interactive Image Generation](https://mlsanity.com/paper/1904.02526/) (2019)
- [Interactive Deep Colorization With Simultaneous Global and Local Inputs](https://mlsanity.com/paper/1801.09083/) (2018)

## Built on by (most cited first, 3 of 3)

- [Image Referenced Sketch Colorization Based on Animation Creation Workflow](https://mlsanity.com/paper/2502.19937/) (2025)
- [ColorFLUX: A Structure-Color Decoupling Framework for Old Photo Colorization](https://mlsanity.com/paper/2603.28162/) (2026)
- [Computer-aided Colorization State-of-the-science: A Survey](https://mlsanity.com/paper/2410.02288/) (2024)

## Abstract (arXiv)

Existing deep interactive colorization models have focused on ways to utilize various types of interactions, such as point-wise color hints, scribbles, or natural-language texts, as methods to reflect a user's intent at runtime. However, another approach, which actively informs the user of the most effective regions to give hints for sketch image colorization, has been under-explored. This paper proposes a novel model-guided deep interactive colorization framework that reduces the required amount of user interactions, by prioritizing the regions in a colorization model. Our method, called GuidingPainter, prioritizes these regions where the model most needs a color hint, rather than just relying on the user's manual decision on where to give a color hint. In our extensive experiments, we show that our approach outperforms existing interactive colorization methods in terms of the conventional metrics, such as PSNR and FID, and reduces required amount of interactions.