# Deep Generation of Face Images from Sketches

arXiv: https://arxiv.org/abs/2006.01047 · HTML: https://mlsanity.com/paper/2006.01047/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2020.

## Built on by (most cited first, 30 of 32)

- [FENeRF: Face Editing in Neural Radiance Fields](https://mlsanity.com/paper/2111.15490/) (2021)
- [Live Speech Portraits: Real-Time Photorealistic Talking-Head Animation](https://mlsanity.com/paper/2109.10595/) (2021)
- [Picture that Sketch: Photorealistic Image Generation from Abstract Sketches](https://mlsanity.com/paper/2303.11162/) (2023)
- [Analogist: Out-of-the-box Visual In-Context Learning with Image Diffusion Model](https://mlsanity.com/paper/2405.10316/) (2024)
- [Towards Open-World Text-Guided Face Image Generation and Manipulation](https://mlsanity.com/paper/2104.08910/) (2021)
- [Fine-grained Image Editing by Pixel-wise Guidance Using Diffusion Models](https://mlsanity.com/paper/2212.02024/) (2022)
- [Image2Lego: Customized LEGO Set Generation from Images](https://mlsanity.com/paper/2108.08477/) (2021)
- [Facial-Sketch Synthesis: A New Challenge](https://mlsanity.com/paper/2112.15439/) (2021)
- [dualFace:Two-Stage Drawing Guidance for Freehand Portrait Sketching](https://mlsanity.com/paper/2104.12297/) (2021)
- [Deep Image Synthesis from Intuitive User Input: A Review and Perspectives](https://mlsanity.com/paper/2107.04240/) (2021)
- [IA-FaceS: A Bidirectional Method for Semantic Face Editing](https://mlsanity.com/paper/2203.13097/) (2022)
- [Content-Based Search for Deep Generative Models](https://mlsanity.com/paper/2210.03116/) (2022)
- [DeepPortraitDrawing: Generating Human Body Images from Freehand Sketches](https://mlsanity.com/paper/2205.02070/) (2022)
- [GANravel: User-Driven Direction Disentanglement in Generative Adversarial Networks](https://mlsanity.com/paper/2302.00079/) (2023)
- [Normal-guided Garment UV Prediction for Human Re-texturing](https://mlsanity.com/paper/2303.06504/) (2023)
- [S2TD-Face: Reconstruct a Detailed 3D Face with Controllable Texture from a Single Sketch](https://mlsanity.com/paper/2408.01218/) (2024)
- [Improving Sketch Colorization using Adversarial Segmentation Consistency](https://mlsanity.com/paper/2301.08590/) (2023)
- [CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing](https://mlsanity.com/paper/2402.17624/) (2024)
- [CraftSVG: Multi-Object Text-to-SVG Synthesis via Layout Guided Diffusion](https://mlsanity.com/paper/2404.00412/) (2024)
- [Locally-Focused Face Representation for Sketch-to-Image Generation Using Noise-Induced Refinement](https://mlsanity.com/paper/2411.19005/) (2024)
- [Adversarial Segmentation Loss for Sketch Colorization](https://mlsanity.com/paper/2102.06192/) (2021)
- [SketchHairSalon: Deep Sketch-based Hair Image Synthesis](https://mlsanity.com/paper/2109.07874/) (2021)
- [MixSyn: Learning Composition and Style for Multi-Source Image Synthesis](https://mlsanity.com/paper/2111.12705/) (2021)
- [Make Your Brief Stroke Real and Stereoscopic: 3D-Aware Simplified Sketch to Portrait Generation](https://mlsanity.com/paper/2302.06857/) (2023)
- [High-Quality Medical Image Generation from Free-hand Sketch](https://mlsanity.com/paper/2402.00353/) (2024)
- [Sketch3DVE: Sketch-based 3D-Aware Scene Video Editing](https://mlsanity.com/paper/2508.13797/) (2025)
- [Deep Generative Framework for Interactive 3D Terrain Authoring and Manipulation](https://mlsanity.com/paper/2201.02369/) (2022)
- [Style-Content Disentanglement in Language-Image Pretraining Representations for Zero-Shot Sketch-to-Image Synthesis](https://mlsanity.com/paper/2206.01661/) (2022)
- [Rethinking the editing of generative adversarial networks: a method to estimate editing vectors based on dimension reduction](https://mlsanity.com/paper/2305.09454/) (2023)
- [CLIPDraw++: Text-to-Sketch Synthesis with Simple Primitives](https://mlsanity.com/paper/2312.02345/) (2023)

## Abstract (arXiv)

Recent deep image-to-image translation techniques allow fast generation of face images from freehand sketches. However, existing solutions tend to overfit to sketches, thus requiring professional sketches or even edge maps as input. To address this issue, our key idea is to implicitly model the shape space of plausible face images and synthesize a face image in this space to approximate an input sketch. We take a local-to-global approach. We first learn feature embeddings of key face components, and push corresponding parts of input sketches towards underlying component manifolds defined by the feature vectors of face component samples. We also propose another deep neural network to learn the mapping from the embedded component features to realistic images with multi-channel feature maps as intermediate results to improve the information flow. Our method essentially uses input sketches as soft constraints and is thus able to produce high-quality face images even from rough and/or incomplete sketches. Our tool is easy to use even for non-artists, while still supporting fine-grained control of shape details. Both qualitative and quantitative evaluations show the superior generation ability of our system to existing and alternative solutions. The usability and expressiveness of our system are confirmed by a user study.