# Effect of Instance Normalization on Fine-Grained Control for Sketch-Based Face Image Generation

arXiv: https://arxiv.org/abs/2207.08072 · HTML: https://mlsanity.com/paper/2207.08072/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 17 Jul 2022.

## Builds on (most cited first, 20 of 20)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [Semantic Image Synthesis via Adversarial Learning](https://mlsanity.com/paper/1707.06873/) (2017)
- [A Style-Aware Content Loss for Real-time HD Style Transfer](https://mlsanity.com/paper/1807.10201/) (2018)
- [SPA-GAN: Spatial Attention GAN for Image-to-Image Translation](https://mlsanity.com/paper/1908.06616/) (2019)
- [TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1904.09571/) (2019)
- [LinesToFacePhoto: Face Photo Generation from Lines with Conditional Self-Attention Generative Adversarial Network](https://mlsanity.com/paper/1910.08914/) (2019)
- [Harmonic Unpaired Image-to-image Translation](https://mlsanity.com/paper/1902.09727/) (2019)
- [CariGANs: Unpaired Photo-to-Caricature Translation](https://mlsanity.com/paper/1811.00222/) (2018)
- [SingleGAN: Image-to-Image Translation by a Single-Generator Network using Multiple Generative Adversarial Learning](https://mlsanity.com/paper/1810.04991/) (2018)

## Abstract (arXiv)

Sketching is an intuitive and effective way for content creation. While significant progress has been made for photorealistic image generation by using generative adversarial networks, it remains challenging to take a fine-grained control on synthetic content. The instance normalization layer, which is widely adopted in existing image translation networks, washes away details in the input sketch and leads to loss of precise control on the desired shape of the generated face images. In this paper, we comprehensively investigate the effect of instance normalization on generating photorealistic face images from hand-drawn sketches. We first introduce a visualization approach to analyze the feature embedding for sketches with a group of specific changes. Based on the visual analysis, we modify the instance normalization layers in the baseline image translation model. We elaborate a new set of hand-drawn sketches with 11 categories of specially designed changes and conduct extensive experimental analysis. The results and user studies demonstrate that our method markedly improve the quality of synthesized images and the conformance with user intention.