# SketchHairSalon: Deep Sketch-based Hair Image Synthesis

arXiv: https://arxiv.org/abs/2109.07874 · HTML: https://mlsanity.com/paper/2109.07874/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 16 Sep 2021.

## Builds on (most cited first, 23 of 23)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [Generating Images with Perceptual Similarity Metrics based on Deep Networks](https://mlsanity.com/paper/1602.02644/) (2016)
- [Fader Networks: Manipulating Images by Sliding Attributes](https://mlsanity.com/paper/1706.00409/) (2017)
- [Scribbler: Controlling Deep Image Synthesis with Sketch and Color](https://mlsanity.com/paper/1612.00835/) (2016)
- [Natural Image Matting via Guided Contextual Attention](https://mlsanity.com/paper/2001.04069/) (2020)
- [Mask-Guided Portrait Editing with Conditional GANs](https://mlsanity.com/paper/1905.10346/) (2019)
- [ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes](https://mlsanity.com/paper/1803.10562/) (2018)
- [Deep Generation of Face Images from Sketches](https://mlsanity.com/paper/2006.01047/) (2020)
- [DeepSketchHair: Deep Sketch-based 3D Hair Modeling](https://mlsanity.com/paper/1908.07198/) (2019)
- [MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing](https://mlsanity.com/paper/2010.16417/) (2020)
- [Intuitive, Interactive Beard and Hair Synthesis with Generative Models](https://mlsanity.com/paper/2004.06848/) (2020)
- [Neural Hair Rendering](https://mlsanity.com/paper/2004.13297/) (2020)
- [DeepFaceEditing: Deep Face Generation and Editing with Disentangled Geometry and Appearance Control](https://mlsanity.com/paper/2105.08935/) (2021)
- [HairNet: Single-View Hair Reconstruction using Convolutional Neural Networks](https://mlsanity.com/paper/1806.07467/) (2018)
- [Two-phase Hair Image Synthesis by Self-Enhancing Generative Model](https://mlsanity.com/paper/1902.11203/) (2019)
- [Continuous Curve Textures](https://mlsanity.com/paper/2012.07959/) (2020)

## Built on by (most cited first, 2 of 2)

- [NeuralHDHair: Automatic High-fidelity Hair Modeling from a Single Image Using Implicit Neural Representations](https://mlsanity.com/paper/2205.04175/) (2022)
- [CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing](https://mlsanity.com/paper/2402.17624/) (2024)

## Abstract (arXiv)

Recent deep generative models allow real-time generation of hair images from sketch inputs. Existing solutions often require a user-provided binary mask to specify a target hair shape. This not only costs users extra labor but also fails to capture complicated hair boundaries. Those solutions usually encode hair structures via orientation maps, which, however, are not very effective to encode complex structures. We observe that colored hair sketches already implicitly define target hair shapes as well as hair appearance and are more flexible to depict hair structures than orientation maps. Based on these observations, we present SketchHairSalon, a two-stage framework for generating realistic hair images directly from freehand sketches depicting desired hair structure and appearance. At the first stage, we train a network to predict a hair matte from an input hair sketch, with an optional set of non-hair strokes. At the second stage, another network is trained to synthesize the structure and appearance of hair images from the input sketch and the generated matte. To make the networks in the two stages aware of long-term dependency of strokes, we apply self-attention modules to them. To train these networks, we present a new dataset containing thousands of annotated hair sketch-image pairs and corresponding hair mattes. Two efficient methods for sketch completion are proposed to automatically complete repetitive braided parts and hair strokes, respectively, thus reducing the workload of users. Based on the trained networks and the two sketch completion strategies, we build an intuitive interface to allow even novice users to design visually pleasing hair images exhibiting various hair structures and appearance via freehand sketches. The qualitative and quantitative evaluations show the advantages of the proposed system over the existing or alternative solutions.