mlsanityData to 7 Oct 2026

SketchFaceGS: Real-Time Sketch-Driven Face Editing and Generation with Gaussian Splatting

arXiv 2604.19202 ↗First posted 21 Apr 2026cs.GR cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

3D Gaussian representations have emerged as a powerful paradigm for digital head modeling, achieving photorealistic quality with real-time rendering. However, intuitive and interactive creation or editing of 3D Gaussian head models remains challenging. Although 2D sketches provide an ideal interaction modality for fast, intuitive conceptual design, they are sparse, depth-ambiguous, and lack high-frequency appearance cues, making it difficult to infer dense, geometrically consistent 3D Gaussian structures from strokes - especially under real-time constraints. To address these challenges, we propose SketchFaceGS, the first sketch-driven framework for real-time generation and editing of photorealistic 3D Gaussian head models from 2D sketches. Our method uses a feed-forward, coarse-to-fine architecture. A Transformer-based UV feature-prediction module first reconstructs a coarse but geometrically consistent UV feature map from the input sketch, and then a 3D UV feature enhancement module refines it with high-frequency, photorealistic detail to produce a high-fidelity 3D head. For editing, we introduce a UV Mask Fusion technique combined with a layer-by-layer feature-fusion strategy, enabling precise, real-time, free-viewpoint modifications. Extensive experiments show that SketchFaceGS outperforms existing methods in both generation fidelity and editing flexibility, producing high-quality, editable 3D heads from sketches in a single forward pass.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

40 of 59 references.

References · 40

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  6. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  7. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  8. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  9. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  10. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  11. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  12. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
28 more
  1. 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 347
  2. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  3. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  4. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  5. 2023GaussianEditor: Editing 3D Gaussians Delicately with Text Instructionscited by 166
  6. 2024Recent Advances in 3D Gaussian Splattingcited by 130
  7. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  8. 2024GaussCtrl: Multi-View Consistent Text-Driven 3D Gaussian Splatting Editingcited by 114
  9. 2019SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Colorcited by 105
  10. 2022Generative Neural Articulated Radiance Fieldscited by 101
  11. 2024Generalizable and Animatable Gaussian Head Avatarcited by 95
  12. 2022GIRAFFE HD: A High-Resolution 3D-aware Generative Modelcited by 86
  13. 20243D Gaussian Blendshapes for Head Avatar Animationcited by 58
  14. 2024HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priorscited by 54
  15. 2024GGHead: Fast and Generalizable 3D Gaussian Headscited by 51
  16. 2020Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketchescited by 37
  17. 2021SketchEdit: Mask-Free Local Image Manipulation with Partial Sketchescited by 32
  18. 2024MagicQuill: An Intelligent Interactive Image Editing Systemcited by 30
  19. 2025RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatarscited by 27
  20. 20233D-aware Conditional Image Synthesiscited by 24
  21. 2019LinesToFacePhoto: Face Photo Generation from Lines with Conditional Self-Attention Generative Adversarial Networkcited by 23
  22. 2024Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRMcited by 23
  23. 2020DeepFacePencil: Creating Face Images from Freehand Sketchescited by 19
  24. 2018CaricatureShop: Personalized and Photorealistic Caricature Sketchingcited by 12
  25. 2025Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPscited by 12
  26. 2021SimpModeling: Sketching Implicit Field to Guide Mesh Modeling for 3D Animalmorphic Head Designcited by 9
  27. 2024GSGAN: Adversarial Learning for Hierarchical Generation of 3D Gaussian Splatscited by 9
  28. 2025S3D: Sketch-Driven 3D Model Generationcited by 1

Built on by

Not cited yet.