SketchFaceGS: Real-Time Sketch-Driven Face Editing and Generation with Gaussian Splatting
Not on the map. Why?
Abstract (arXiv)
3D Gaussian representations have emerged as a powerful paradigm for digital head modeling, achieving photorealistic quality with real-time rendering. However, intuitive and interactive creation or editing of 3D Gaussian head models remains challenging. Although 2D sketches provide an ideal interaction modality for fast, intuitive conceptual design, they are sparse, depth-ambiguous, and lack high-frequency appearance cues, making it difficult to infer dense, geometrically consistent 3D Gaussian structures from strokes - especially under real-time constraints. To address these challenges, we propose SketchFaceGS, the first sketch-driven framework for real-time generation and editing of photorealistic 3D Gaussian head models from 2D sketches. Our method uses a feed-forward, coarse-to-fine architecture. A Transformer-based UV feature-prediction module first reconstructs a coarse but geometrically consistent UV feature map from the input sketch, and then a 3D UV feature enhancement module refines it with high-frequency, photorealistic detail to produce a high-fidelity 3D head. For editing, we introduce a UV Mask Fusion technique combined with a layer-by-layer feature-fusion strategy, enabling precise, real-time, free-viewpoint modifications. Extensive experiments show that SketchFaceGS outperforms existing methods in both generation fidelity and editing flexibility, producing high-quality, editable 3D heads from sketches in a single forward pass.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 21 Apr 2026.
Builds on
40 of 59 references.
References · 40
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
- 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
- 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
- 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
- 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
- 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
28 more
- 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 347
- 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
- 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
- 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
- 2023GaussianEditor: Editing 3D Gaussians Delicately with Text Instructionscited by 166
- 2024Recent Advances in 3D Gaussian Splattingcited by 130
- 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
- 2024GaussCtrl: Multi-View Consistent Text-Driven 3D Gaussian Splatting Editingcited by 114
- 2019SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Colorcited by 105
- 2022Generative Neural Articulated Radiance Fieldscited by 101
- 2024Generalizable and Animatable Gaussian Head Avatarcited by 95
- 2022GIRAFFE HD: A High-Resolution 3D-aware Generative Modelcited by 86
- 20243D Gaussian Blendshapes for Head Avatar Animationcited by 58
- 2024HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priorscited by 54
- 2024GGHead: Fast and Generalizable 3D Gaussian Headscited by 51
- 2020Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketchescited by 37
- 2021SketchEdit: Mask-Free Local Image Manipulation with Partial Sketchescited by 32
- 2024MagicQuill: An Intelligent Interactive Image Editing Systemcited by 30
- 2025RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatarscited by 27
- 20233D-aware Conditional Image Synthesiscited by 24
- 2019LinesToFacePhoto: Face Photo Generation from Lines with Conditional Self-Attention Generative Adversarial Networkcited by 23
- 2024Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRMcited by 23
- 2020DeepFacePencil: Creating Face Images from Freehand Sketchescited by 19
- 2018CaricatureShop: Personalized and Photorealistic Caricature Sketchingcited by 12
- 2025Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPscited by 12
- 2021SimpModeling: Sketching Implicit Field to Guide Mesh Modeling for 3D Animalmorphic Head Designcited by 9
- 2024GSGAN: Adversarial Learning for Hierarchical Generation of 3D Gaussian Splatscited by 9
- 2025S3D: Sketch-Driven 3D Model Generationcited by 1
Built on by
Not cited yet.