# DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling

arXiv: https://arxiv.org/abs/1706.02042 · HTML: https://mlsanity.com/paper/1706.02042/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2017.

## Built on by (most cited first, 16 of 16)

- [Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketches](https://mlsanity.com/paper/2105.06663/) (2021)
- [Neural Cloth Simulation](https://mlsanity.com/paper/2212.11220/) (2022)
- [RaBit: Parametric Modeling of 3D Biped Cartoon Characters with a Topological-consistent Dataset](https://mlsanity.com/paper/2303.12564/) (2023)
- [Deep Deformable 3D Caricatures with Learned Shape Control](https://mlsanity.com/paper/2207.14593/) (2022)
- [AgileAvatar: Stylized 3D Avatar Creation via Cascaded Domain Bridging](https://mlsanity.com/paper/2211.07818/) (2022)
- [SimpModeling: Sketching Implicit Field to Guide Mesh Modeling for 3D Animalmorphic Head Design](https://mlsanity.com/paper/2108.02548/) (2021)
- [Learning Geometry-aware Representations by Sketching](https://mlsanity.com/paper/2304.08204/) (2023)
- [Sketch-based Normal Map Generation with Geometric Sampling](https://mlsanity.com/paper/2104.11554/) (2021)
- [Deep Sketch-Based 3D Modeling: A Survey](https://mlsanity.com/paper/2603.03287/) (2026)
- [DifferSketching: How Differently Do People Sketch 3D Objects?](https://mlsanity.com/paper/2209.08791/) (2022)
- [Tag-based annotation creates better avatars](https://mlsanity.com/paper/2302.07354/) (2023)
- [Sketch3DVE: Sketch-based 3D-Aware Scene Video Editing](https://mlsanity.com/paper/2508.13797/) (2025)
- [KD360-VoxelBEV: LiDAR and 360-degree Camera Cross Modality Knowledge Distillation for Bird's-Eye-View Segmentation](https://mlsanity.com/paper/2512.15311/) (2025)
- [3Deformer: A Common Framework for Image-Guided Mesh Deformation](https://mlsanity.com/paper/2307.09892/) (2023)
- [Sketch-1-to-3: One Single Sketch to 3D Detailed Face Reconstruction](https://mlsanity.com/paper/2502.17852/) (2025)
- [CaricatureGS: Exaggerating 3D Gaussian Splatting Faces With Gaussian Curvature](https://mlsanity.com/paper/2601.03319/) (2026)

## Abstract (arXiv)

Face modeling has been paid much attention in the field of visual computing. There exist many scenarios, including cartoon characters, avatars for social media, 3D face caricatures as well as face-related art and design, where low-cost interactive face modeling is a popular approach especially among amateur users. In this paper, we propose a deep learning based sketching system for 3D face and caricature modeling. This system has a labor-efficient sketching interface, that allows the user to draw freehand imprecise yet expressive 2D lines representing the contours of facial features. A novel CNN based deep regression network is designed for inferring 3D face models from 2D sketches. Our network fuses both CNN and shape based features of the input sketch, and has two independent branches of fully connected layers generating independent subsets of coefficients for a bilinear face representation. Our system also supports gesture based interactions for users to further manipulate initial face models. Both user studies and numerical results indicate that our sketching system can help users create face models quickly and effectively. A significantly expanded face database with diverse identities, expressions and levels of exaggeration is constructed to promote further research and evaluation of face modeling techniques.