# AniFaceDrawing: Anime Portrait Exploration during Your Sketching

arXiv: https://arxiv.org/abs/2306.07476 · HTML: https://mlsanity.com/paper/2306.07476/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 13 Jun 2023.

## Builds on (most cited first, 19 of 19)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment](https://mlsanity.com/paper/1908.06391/) (2019)
- [InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs](https://mlsanity.com/paper/2005.09635/) (2020)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement](https://mlsanity.com/paper/2104.02699/) (2021)
- [Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval](https://mlsanity.com/paper/1703.05605/) (2017)
- [Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer](https://mlsanity.com/paper/2203.13248/) (2022)
- [Interactive Sketch & Fill: Multiclass Sketch-to-Image Translation](https://mlsanity.com/paper/1909.11081/) (2019)
- [DeepWriting: Making Digital Ink Editable via Deep Generative Modeling](https://mlsanity.com/paper/1801.08379/) (2018)
- [Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches](https://mlsanity.com/paper/2001.02890/) (2020)
- [LiveSketch: Query Perturbations for Guided Sketch-based Visual Search](https://mlsanity.com/paper/1904.06611/) (2019)
- [Smart, Sparse Contours to Represent and Edit Images](https://mlsanity.com/paper/1712.08232/) (2017)
- [DeepFacePencil: Creating Face Images from Freehand Sketches](https://mlsanity.com/paper/2008.13343/) (2020)
- [DrawingInStyles: Portrait Image Generation and Editing with Spatially Conditioned StyleGAN](https://mlsanity.com/paper/2203.02762/) (2022)
- [Disentangled Face Attribute Editing via Instance-Aware Latent Space Search](https://mlsanity.com/paper/2105.12660/) (2021)
- [Modeling Artistic Workflows for Image Generation and Editing](https://mlsanity.com/paper/2007.07238/) (2020)

## Built on by (most cited first, 7 of 7)

- [It's All About Your Sketch: Democratising Sketch Control in Diffusion Models](https://mlsanity.com/paper/2403.07234/) (2024)
- [APISR: Anime Production Inspired Real-World Anime Super-Resolution](https://mlsanity.com/paper/2403.01598/) (2024)
- [Generative AI Models for Different Steps in Architectural Design: A Literature Review](https://mlsanity.com/paper/2404.01335/) (2024)
- [Sketch-Guided Scene Image Generation](https://mlsanity.com/paper/2407.06469/) (2024)
- [Sketch-Guided Stylized Landscape Cinemagraph Synthesis](https://mlsanity.com/paper/2412.00638/) (2024)
- [Interactive Drawing Guidance for Anime Illustrations with Diffusion Model](https://mlsanity.com/paper/2507.09140/) (2025)
- [No Pixel Left Behind: Filling Gaps in Anime Colorization](https://mlsanity.com/paper/2609.00800/) (2026)

## Abstract (arXiv)

In this paper, we focus on how artificial intelligence (AI) can be used to assist users in the creation of anime portraits, that is, converting rough sketches into anime portraits during their sketching process. The input is a sequence of incomplete freehand sketches that are gradually refined stroke by stroke, while the output is a sequence of high-quality anime portraits that correspond to the input sketches as guidance. Although recent GANs can generate high quality images, it is a challenging problem to maintain the high quality of generated images from sketches with a low degree of completion due to ill-posed problems in conditional image generation. Even with the latest sketch-to-image (S2I) technology, it is still difficult to create high-quality images from incomplete rough sketches for anime portraits since anime style tend to be more abstract than in realistic style. To address this issue, we adopt a latent space exploration of StyleGAN with a two-stage training strategy. We consider the input strokes of a freehand sketch to correspond to edge information-related attributes in the latent structural code of StyleGAN, and term the matching between strokes and these attributes stroke-level disentanglement. In the first stage, we trained an image encoder with the pre-trained StyleGAN model as a teacher encoder. In the second stage, we simulated the drawing process of the generated images without any additional data (labels) and trained the sketch encoder for incomplete progressive sketches to generate high-quality portrait images with feature alignment to the disentangled representations in the teacher encoder. We verified the proposed progressive S2I system with both qualitative and quantitative evaluations and achieved high-quality anime portraits from incomplete progressive sketches. Our user study proved its effectiveness in art creation assistance for the anime style.