# inkn'hue: Enhancing Manga Colorization from Multiple Priors with Alignment Multi-Encoder VAE

arXiv: https://arxiv.org/abs/2311.01804 · HTML: https://mlsanity.com/paper/2311.01804/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 3 Nov 2023.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [An Introduction to Convolutional Neural Networks](https://mlsanity.com/paper/1511.08458/) (2015)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks](https://mlsanity.com/paper/1604.04382/) (2016)
- [Denoising Criterion for Variational Auto-Encoding Framework](https://mlsanity.com/paper/1511.06406/) (2015)
- [Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss](https://mlsanity.com/paper/1908.05840/) (2019)
- [User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks](https://mlsanity.com/paper/1808.03240/) (2018)
- [Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN](https://mlsanity.com/paper/1706.03319/) (2017)
- [Improved Diffusion-based Image Colorization via Piggybacked Models](https://mlsanity.com/paper/2304.11105/) (2023)
- [cGAN-based Manga Colorization Using a Single Training Image](https://mlsanity.com/paper/1706.06918/) (2017)
- [Painting Style-Aware Manga Colorization Based on Generative Adversarial Networks](https://mlsanity.com/paper/2107.07943/) (2021)
- [Line Art Colorization of Fakemon using Generative Adversarial Neural Networks](https://mlsanity.com/paper/2307.05760/) (2023)

## Built on by (most cited first, 1 of 1)

- [One missing piece in Vision and Language: A Survey on Comics Understanding](https://mlsanity.com/paper/2409.09502/) (2024)

## Abstract (arXiv)

Manga, a form of Japanese comics and distinct visual storytelling, has captivated readers worldwide. Traditionally presented in black and white, manga's appeal lies in its ability to convey complex narratives and emotions through intricate line art and shading. Yet, the desire to experience manga in vibrant colors has sparked the pursuit of manga colorization, a task of paramount significance for artists. However, existing methods, originally designed for line art and sketches, face challenges when applied to manga. These methods often fall short in achieving the desired results, leading to the need for specialized manga-specific solutions. Existing approaches frequently rely on a single training step or extensive manual artist intervention, which can yield less satisfactory outcomes. To address these challenges, we propose a specialized framework for manga colorization. Leveraging established models for shading and vibrant coloring, our approach aligns both using a multi-encoder VAE. This structured workflow ensures clear and colorful results, with the option to incorporate reference images and manual hints.