# Sketch-1-to-3: One Single Sketch to 3D Detailed Face Reconstruction

arXiv: https://arxiv.org/abs/2502.17852 · HTML: https://mlsanity.com/paper/2502.17852/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 51 references that are in the map.
- First posted on arXiv 25 Feb 2025.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [Pixel Difference Networks for Efficient Edge Detection](https://mlsanity.com/paper/2108.07009/) (2021)
- [SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color](https://mlsanity.com/paper/1902.06838/) (2019)
- [Unsupervised Training for 3D Morphable Model Regression](https://mlsanity.com/paper/1806.06098/) (2018)
- [On Face Segmentation, Face Swapping, and Face Perception](https://mlsanity.com/paper/1704.06729/) (2017)
- [MixStyle Neural Networks for Domain Generalization and Adaptation](https://mlsanity.com/paper/2107.02053/) (2021)
- [Towards a complete 3D morphable model of the human head](https://mlsanity.com/paper/1911.08008/) (2019)
- [Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation](https://mlsanity.com/paper/1703.10131/) (2017)
- [Identity-Aware CycleGAN for Face Photo-Sketch Synthesis and Recognition](https://mlsanity.com/paper/2103.16019/) (2021)
- [SCUT-FBP5500: A Diverse Benchmark Dataset for Multi-Paradigm Facial Beauty Prediction](https://mlsanity.com/paper/1801.06345/) (2018)
- [DeepFacePencil: Creating Face Images from Freehand Sketches](https://mlsanity.com/paper/2008.13343/) (2020)
- [DrawingInStyles: Portrait Image Generation and Editing with Spatially Conditioned StyleGAN](https://mlsanity.com/paper/2203.02762/) (2022)
- [DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling](https://mlsanity.com/paper/1706.02042/) (2017)
- [Fitting 3D Morphable Models using Local Features](https://mlsanity.com/paper/1503.02330/) (2015)
- [S2TD-Face: Reconstruct a Detailed 3D Face with Controllable Texture from a Single Sketch](https://mlsanity.com/paper/2408.01218/) (2024)
- [Audio-Driven Talking Face Generation with Diverse yet Realistic Facial Animations](https://mlsanity.com/paper/2304.08945/) (2023)
- [SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling](https://mlsanity.com/paper/2307.00804/) (2023)
- [SPColor: Semantic Prior Guided Exemplar-based Image Colorization](https://mlsanity.com/paper/2304.06255/) (2023)
- [BabyNet: Reconstructing 3D faces of babies from uncalibrated photographs](https://mlsanity.com/paper/2203.05908/) (2022)

## Abstract (arXiv)

3D face reconstruction from a single sketch is a critical yet underexplored task with significant practical applications. The primary challenges stem from the substantial modality gap between 2D sketches and 3D facial structures, including: (1) accurately extracting facial keypoints from 2D sketches; (2) preserving diverse facial expressions and fine-grained texture details; and (3) training a high-performing model with limited data. In this paper, we propose Sketch-1-to-3, a novel framework for realistic 3D face reconstruction from a single sketch, to address these challenges. Specifically, we first introduce the Geometric Contour and Texture Detail (GCTD) module, which enhances the extraction of geometric contours and texture details from facial sketches. Additionally, we design a deep learning architecture with a domain adaptation module and a tailored loss function to align sketches with the 3D facial space, enabling high-fidelity expression and texture reconstruction. To facilitate evaluation and further research, we construct SketchFaces, a real hand-drawn facial sketch dataset, and Syn-SketchFaces, a synthetic facial sketch dataset. Extensive experiments demonstrate that Sketch-1-to-3 achieves state-of-the-art performance in sketch-based 3D face reconstruction.