# Deep Exemplar-based Colorization

arXiv: https://arxiv.org/abs/1807.06587 · HTML: https://mlsanity.com/paper/1807.06587/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2018.

## Built on by (most cited first, 30 of 82)

- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models](https://mlsanity.com/paper/2308.07863/) (2023)
- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences](https://mlsanity.com/paper/2108.07422/) (2021)
- [PISE: Person Image Synthesis and Editing with Decoupled GAN](https://mlsanity.com/paper/2103.04023/) (2021)
- [Towards Vivid and Diverse Image Colorization with Generative Color Prior](https://mlsanity.com/paper/2108.08826/) (2021)
- [Unbalanced Feature Transport for Exemplar-based Image Translation](https://mlsanity.com/paper/2106.10482/) (2021)
- [DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders](https://mlsanity.com/paper/2212.11613/) (2022)
- [Control Color: Multimodal Diffusion-based Interactive Image Colorization](https://mlsanity.com/paper/2402.10855/) (2024)
- [Bringing Old Films Back to Life](https://mlsanity.com/paper/2203.17276/) (2022)
- [Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning](https://mlsanity.com/paper/2110.04562/) (2021)
- [SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal](https://mlsanity.com/paper/2112.03631/) (2021)
- [Bi-level Feature Alignment for Versatile Image Translation and Manipulation](https://mlsanity.com/paper/2107.03021/) (2021)
- [PalGAN: Image Colorization with Palette Generative Adversarial Networks](https://mlsanity.com/paper/2210.11204/) (2022)
- [Improved Diffusion-based Image Colorization via Piggybacked Models](https://mlsanity.com/paper/2304.11105/) (2023)
- [BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization](https://mlsanity.com/paper/2212.02268/) (2022)
- [Deep Edge-Aware Interactive Colorization against Color-Bleeding Effects](https://mlsanity.com/paper/2107.01619/) (2021)
- [Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization](https://mlsanity.com/paper/2112.01335/) (2021)
- [Video Colorization with Pre-trained Text-to-Image Diffusion Models](https://mlsanity.com/paper/2306.01732/) (2023)
- [Color2Embed: Fast Exemplar-Based Image Colorization using Color Embeddings](https://mlsanity.com/paper/2106.08017/) (2021)
- [Self-supervised Correlation Mining Network for Person Image Generation](https://mlsanity.com/paper/2111.13307/) (2021)
- [SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution](https://mlsanity.com/paper/2201.04364/) (2022)
- [Pik-Fix: Restoring and Colorizing Old Photos](https://mlsanity.com/paper/2205.01902/) (2022)
- [ColorizeDiffusion: Adjustable Sketch Colorization with Reference Image and Text](https://mlsanity.com/paper/2401.01456/) (2024)
- [ColorMNet: A Memory-based Deep Spatial-Temporal Feature Propagation Network for Video Colorization](https://mlsanity.com/paper/2404.06251/) (2024)
- [Makeup-Guided Facial Privacy Protection via Untrained Neural Network Priors](https://mlsanity.com/paper/2408.12387/) (2024)
- [UniColor: A Unified Framework for Multi-Modal Colorization with Transformer](https://mlsanity.com/paper/2209.11223/) (2022)
- [Automatic Controllable Colorization via Imagination](https://mlsanity.com/paper/2404.05661/) (2024)
- [Neural Photometry-guided Visual Attribute Transfer](https://mlsanity.com/paper/2112.02520/) (2021)
- [Controlled and Conditional Text to Image Generation with Diffusion Prior](https://mlsanity.com/paper/2302.11710/) (2023)

## Abstract (arXiv)

We propose the first deep learning approach for exemplar-based local colorization. Given a reference color image, our convolutional neural network directly maps a grayscale image to an output colorized image. Rather than using hand-crafted rules as in traditional exemplar-based methods, our end-to-end colorization network learns how to select, propagate, and predict colors from the large-scale data. The approach performs robustly and generalizes well even when using reference images that are unrelated to the input grayscale image. More importantly, as opposed to other learning-based colorization methods, our network allows the user to achieve customizable results by simply feeding different references. In order to further reduce manual effort in selecting the references, the system automatically recommends references with our proposed image retrieval algorithm, which considers both semantic and luminance information. The colorization can be performed fully automatically by simply picking the top reference suggestion. Our approach is validated through a user study and favorable quantitative comparisons to the-state-of-the-art methods. Furthermore, our approach can be naturally extended to video colorization. Our code and models will be freely available for public use.