# Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence

arXiv: https://arxiv.org/abs/2005.05207 · HTML: https://mlsanity.com/paper/2005.05207/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2020.

## Built on by (most cited first, 30 of 53)

- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence](https://mlsanity.com/paper/2311.17034/) (2023)
- [Towards Vivid and Diverse Image Colorization with Generative Color Prior](https://mlsanity.com/paper/2108.08826/) (2021)
- [Learning Semantic Correspondence with Sparse Annotations](https://mlsanity.com/paper/2208.06974/) (2022)
- [Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences](https://mlsanity.com/paper/2203.04279/) (2022)
- [Eliminating Gradient Conflict in Reference-based Line-Art Colorization](https://mlsanity.com/paper/2207.06095/) (2022)
- [DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation](https://mlsanity.com/paper/2207.06124/) (2022)
- [Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning](https://mlsanity.com/paper/2110.04562/) (2021)
- [SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal](https://mlsanity.com/paper/2112.03631/) (2021)
- [GAN-based Facial Attribute Manipulation](https://mlsanity.com/paper/2210.12683/) (2022)
- [Learning Inclusion Matching for Animation Paint Bucket Colorization](https://mlsanity.com/paper/2403.18342/) (2024)
- [Can Visual Foundation Models Achieve Long-term Point Tracking?](https://mlsanity.com/paper/2408.13575/) (2024)
- [MOSAIC: Multi-Subject Personalized Generation via Correspondence-Aware Alignment and Disentanglement](https://mlsanity.com/paper/2509.01977/) (2025)
- [Deep Edge-Aware Interactive Colorization against Color-Bleeding Effects](https://mlsanity.com/paper/2107.01619/) (2021)
- [Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization](https://mlsanity.com/paper/2112.01335/) (2021)
- [AnimeCeleb: Large-Scale Animation CelebHeads Dataset for Head Reenactment](https://mlsanity.com/paper/2111.07640/) (2021)
- [Follow-Your-Color: Multi-Instance Sketch Colorization](https://mlsanity.com/paper/2503.16948/) (2025)
- [Color2Embed: Fast Exemplar-Based Image Colorization using Color Embeddings](https://mlsanity.com/paper/2106.08017/) (2021)
- [Self-supervised Correlation Mining Network for Person Image Generation](https://mlsanity.com/paper/2111.13307/) (2021)
- [AnimeDiffusion: Anime Face Line Drawing Colorization via Diffusion Models](https://mlsanity.com/paper/2303.11137/) (2023)
- [SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution](https://mlsanity.com/paper/2201.04364/) (2022)
- [ColorizeDiffusion: Adjustable Sketch Colorization with Reference Image and Text](https://mlsanity.com/paper/2401.01456/) (2024)
- [SemAlign3D: Semantic Correspondence between RGB-Images through Aligning 3D Object-Class Representations](https://mlsanity.com/paper/2503.22462/) (2025)
- [Paint2Pix: Interactive Painting based Progressive Image Synthesis and Editing](https://mlsanity.com/paper/2208.08092/) (2022)
- [ColorFlow: Retrieval-Augmented Image Sequence Colorization](https://mlsanity.com/paper/2412.11815/) (2024)
- [Attention-Aware Anime Line Drawing Colorization](https://mlsanity.com/paper/2212.10988/) (2022)
- [Semi-supervised reference-based sketch extraction using a contrastive learning framework](https://mlsanity.com/paper/2407.14026/) (2024)
- [Image Referenced Sketch Colorization Based on Animation Creation Workflow](https://mlsanity.com/paper/2502.19937/) (2025)
- [Stylized Face Sketch Extraction via Generative Prior with Limited Data](https://mlsanity.com/paper/2403.11263/) (2024)
- [ShadowHack: Hacking Shadows via Luminance-Color Divide and Conquer](https://mlsanity.com/paper/2412.02545/) (2024)

## Abstract (arXiv)

This paper tackles the automatic colorization task of a sketch image given an already-colored reference image. Colorizing a sketch image is in high demand in comics, animation, and other content creation applications, but it suffers from information scarcity of a sketch image. To address this, a reference image can render the colorization process in a reliable and user-driven manner. However, it is difficult to prepare for a training data set that has a sufficient amount of semantically meaningful pairs of images as well as the ground truth for a colored image reflecting a given reference (e.g., coloring a sketch of an originally blue car given a reference green car). To tackle this challenge, we propose to utilize the identical image with geometric distortion as a virtual reference, which makes it possible to secure the ground truth for a colored output image. Furthermore, it naturally provides the ground truth for dense semantic correspondence, which we utilize in our internal attention mechanism for color transfer from reference to sketch input. We demonstrate the effectiveness of our approach in various types of sketch image colorization via quantitative as well as qualitative evaluation against existing methods.