mlsanityData to 14 Oct 2026

Human-Inspired Facial Sketch Synthesis with Dynamic Adaptation

arXiv 2309.00216 ↗First posted 1 Sep 2023cs.CV cs.MMcited by 4

Not on the map. Why?

Abstract (arXiv)

Facial sketch synthesis (FSS) aims to generate a vivid sketch portrait from a given facial photo. Existing FSS methods merely rely on 2D representations of facial semantic or appearance. However, professional human artists usually use outlines or shadings to covey 3D geometry. Thus facial 3D geometry (e.g. depth map) is extremely important for FSS. Besides, different artists may use diverse drawing techniques and create multiple styles of sketches; but the style is globally consistent in a sketch. Inspired by such observations, in this paper, we propose a novel Human-Inspired Dynamic Adaptation (HIDA) method. Specially, we propose to dynamically modulate neuron activations based on a joint consideration of both facial 3D geometry and 2D appearance, as well as globally consistent style control. Besides, we use deformable convolutions at coarse-scales to align deep features, for generating abstract and distinct outlines. Experiments show that HIDA can generate high-quality sketches in multiple styles, and significantly outperforms previous methods, over a large range of challenging faces. Besides, HIDA allows precise style control of the synthesized sketch, and generalizes well to natural scenes and other artistic styles. Our code and results have been released online at: https://github.com/AiArt-HDU/HIDA.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 33 of its 49 references that are in the map. First posted on arXiv 1 Sep 2023.

Builds on

33 of 49 references.

References · 33

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  4. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  5. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  6. 2017Deformable Convolutional Networkscited by 1,488
  7. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  8. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  9. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  10. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  11. 2020U2-Net: Going Deeper with Nested U-Structure for Salient Object Detectioncited by 412
  12. 2021Dynamic Neural Networks: A Surveycited by 408
21 more
  1. 2019SEAN: Image Synthesis with Semantic Region-Adaptive Normalizationcited by 253
  2. 2020Towards Fast, Accurate and Stable 3D Dense Face Alignmentcited by 219
  3. 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
  4. 2020A Review on Generative Adversarial Networks: Algorithms, Theory, and Applicationscited by 209
  5. 2020Cross-domain Correspondence Learning for Exemplar-based Image Translationcited by 116
  6. 2022Learning to generate line drawings that convey geometry and semanticscited by 83
  7. 2021FaPN: Feature-aligned Pyramid Network for Dense Image Predictioncited by 81
  8. 2021Understanding and Creating Art with AI: Review and Outlookcited by 60
  9. 2020Rethinking the Truly Unsupervised Image-to-Image Translationcited by 56
  10. 2020Dynamic ReLUcited by 55
  11. 2021Diverse Semantic Image Synthesis via Probability Distribution Modelingcited by 49
  12. 2021Learning Semantic Person Image Generation by Region-Adaptive Normalizationcited by 44
  13. 2021Unbalanced Feature Transport for Exemplar-based Image Translationcited by 41
  14. 2022DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generationcited by 24
  15. 2020Activate or Not: Learning Customized Activationcited by 22
  16. 2019Im2Pencil: Controllable Pencil Illustration from Photographscited by 18
  17. 2020Domain-Specific Mappings for Generative Adversarial Style Transfercited by 16
  18. 2019Scoot: A Perceptual Metric for Facial Sketchescited by 10
  19. 2018Semi-Supervised Learning for Face Sketch Synthesis in the Wildcited by 7
  20. 2021Unconstrained Face Sketch Synthesis via Perception-Adaptive Network and A New Benchmarkcited by 5
  21. 2022Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network with Graph Representation Learningcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2024CLIP4Sketch: Enhancing Sketch to Mugshot Matching through Dataset Augmentation using Diffusion Modelscited by 2
  2. 2024Multi-Style Facial Sketch Synthesis through Masked Generative Modelingcited by 0
  3. 2024PS-StyleGAN: Illustrative Portrait Sketching using Attention-Based Style Adaptationcited by 0
  4. 2025One-shot Face Sketch Synthesis in the Wild via Generative Diffusion Prior and Instruction Tuningcited by 0