mlsanityData to 7 Oct 2026

Effect of Instance Normalization on Fine-Grained Control for Sketch-Based Face Image Generation

arXiv 2207.08072 ↗First posted 17 Jul 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Sketching is an intuitive and effective way for content creation. While significant progress has been made for photorealistic image generation by using generative adversarial networks, it remains challenging to take a fine-grained control on synthetic content. The instance normalization layer, which is widely adopted in existing image translation networks, washes away details in the input sketch and leads to loss of precise control on the desired shape of the generated face images. In this paper, we comprehensively investigate the effect of instance normalization on generating photorealistic face images from hand-drawn sketches. We first introduce a visualization approach to analyze the feature embedding for sketches with a group of specific changes. Based on the visual analysis, we modify the instance normalization layers in the baseline image translation model. We elaborate a new set of hand-drawn sketches with 11 categories of specially designed changes and conduct extensive experimental analysis. The results and user studies demonstrate that our method markedly improve the quality of synthesized images and the conformance with user intention.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 26 references that are in the map. First posted on arXiv 17 Jul 2022.

Builds on

20 of 26 references.

References · 20

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2016Layer Normalizationcited by 5,579
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2018Group Normalizationcited by 1,514
  6. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  7. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  8. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  9. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  10. 2017Toward Multimodal Image-to-Image Translationcited by 295
  11. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
  12. 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
8 more
  1. 2017Semantic Image Synthesis via Adversarial Learningcited by 87
  2. 2018A Style-Aware Content Loss for Real-time HD Style Transfercited by 58
  3. 2019SPA-GAN: Spatial Attention GAN for Image-to-Image Translationcited by 43
  4. 2019TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translationcited by 40
  5. 2019LinesToFacePhoto: Face Photo Generation from Lines with Conditional Self-Attention Generative Adversarial Networkcited by 23
  6. 2019Harmonic Unpaired Image-to-image Translationcited by 18
  7. 2018CariGANs: Unpaired Photo-to-Caricature Translationcited by 11
  8. 2018SingleGAN: Image-to-Image Translation by a Single-Generator Network using Multiple Generative Adversarial Learningcited by 7

Built on by

Not cited yet.