# Unconstrained Face Sketch Synthesis via Perception-Adaptive Network and A New Benchmark

arXiv: https://arxiv.org/abs/2112.01019 · HTML: https://mlsanity.com/paper/2112.01019/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 86 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Suppressing Uncertainties for Large-Scale Facial Expression Recognition](https://mlsanity.com/paper/2002.10392/) (2020)
- [Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition](https://mlsanity.com/paper/2009.09450/) (2020)
- [Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition](https://mlsanity.com/paper/1904.09939/) (2019)
- [Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting](https://mlsanity.com/paper/2012.04529/) (2020)
- [Crowd Counting with Deep Structured Scale Integration Network](https://mlsanity.com/paper/1908.08692/) (2019)
- [Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning](https://mlsanity.com/paper/2008.00923/) (2020)
- [Contextualized Spatial-Temporal Network for Taxi Origin-Destination Demand Prediction](https://mlsanity.com/paper/1905.06335/) (2019)
- [LARNet: Lie Algebra Residual Network for Face Recognition](https://mlsanity.com/paper/2103.08147/) (2021)
- [End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning](https://mlsanity.com/paper/1501.07180/) (2015)
- [Semi-Supervised Learning for Face Sketch Synthesis in the Wild](https://mlsanity.com/paper/1812.04929/) (2018)
- [Face Sketch Synthesis with Style Transfer using Pyramid Column Feature](https://mlsanity.com/paper/2009.08679/) (2020)
- [Face Sketch Synthesis Style Similarity:A New Structure Co-occurrence Texture Measure](https://mlsanity.com/paper/1804.02975/) (2018)
- [Contour Loss: Boundary-Aware Learning for Salient Object Segmentation](https://mlsanity.com/paper/1908.01975/) (2019)
- [Multi-Margin based Decorrelation Learning for Heterogeneous Face Recognition](https://mlsanity.com/paper/2005.11945/) (2020)
- [DISC: Deep Image Saliency Computing via Progressive Representation Learning](https://mlsanity.com/paper/1511.04192/) (2015)
- [Random Sampling for Fast Face Sketch Synthesis](https://mlsanity.com/paper/1701.01911/) (2017)
- [Content-Adaptive Sketch Portrait Generation by Decompositional Representation Learning](https://mlsanity.com/paper/1710.01453/) (2017)
- [Facial Landmark Machines: A Backbone-Branches Architecture with Progressive Representation Learning](https://mlsanity.com/paper/1812.03887/) (2018)
- [Fast Preprocessing for Robust Face Sketch Synthesis](https://mlsanity.com/paper/1708.00224/) (2017)

## Built on by (most cited first, 5 of 5)

- [Human-Inspired Facial Sketch Synthesis with Dynamic Adaptation](https://mlsanity.com/paper/2309.00216/) (2023)
- [Semi-supervised Cycle-GAN for face photo-sketch translation in the wild](https://mlsanity.com/paper/2307.10281/) (2023)
- [CLIP4Sketch: Enhancing Sketch to Mugshot Matching through Dataset Augmentation using Diffusion Models](https://mlsanity.com/paper/2408.01233/) (2024)
- [Few-shot Face Image Translation via GAN Prior Distillation](https://mlsanity.com/paper/2301.12257/) (2023)
- [One-shot Face Sketch Synthesis in the Wild via Generative Diffusion Prior and Instruction Tuning](https://mlsanity.com/paper/2506.15312/) (2025)

## Abstract (arXiv)

Face sketch generation has attracted much attention in the field of visual computing. However, existing methods either are limited to constrained conditions or heavily rely on various preprocessing steps to deal with in-the-wild cases. In this paper, we argue that accurately perceiving facial region and facial components is crucial for unconstrained sketch synthesis. To this end, we propose a novel Perception-Adaptive Network (PANet), which can generate high-quality face sketches under unconstrained conditions in an end-to-end scheme. Specifically, our PANet is composed of i) a Fully Convolutional Encoder for hierarchical feature extraction, ii) a Face-Adaptive Perceiving Decoder for extracting potential facial region and handling face variations, and iii) a Component-Adaptive Perceiving Module for facial component aware feature representation learning. To facilitate further researches of unconstrained face sketch synthesis, we introduce a new benchmark termed WildSketch, which contains 800 pairs of face photo-sketch with large variations in pose, expression, ethnic origin, background, and illumination. Extensive experiments demonstrate that the proposed method is capable of achieving state-of-the-art performance under both constrained and unconstrained conditions. Our source codes and the WildSketch benchmark are resealed on the project page http://lingboliu.com/unconstrained_face_sketch.html.