# ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions

arXiv: https://arxiv.org/abs/2507.10542 · HTML: https://mlsanity.com/paper/2507.10542/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 14 Jul 2025.

## Builds on (most cited first, 14 of 14)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians](https://mlsanity.com/paper/2312.02069/) (2023)
- [NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads](https://mlsanity.com/paper/2305.03027/) (2023)
- [Real-Time High-Resolution Background Matting](https://mlsanity.com/paper/2012.07810/) (2020)
- [Morphable Face Models - An Open Framework](https://mlsanity.com/paper/1709.08398/) (2017)
- [Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video](https://mlsanity.com/paper/2210.06108/) (2022)
- [AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxels](https://mlsanity.com/paper/2211.13206/) (2022)
- [NPGA: Neural Parametric Gaussian Avatars](https://mlsanity.com/paper/2405.19331/) (2024)
- [MonoNPHM: Dynamic Head Reconstruction from Monocular Videos](https://mlsanity.com/paper/2312.06740/) (2023)
- [Multilinear Wavelets: A Statistical Shape Space for Human Faces](https://mlsanity.com/paper/1401.2818/) (2014)
- [GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations](https://mlsanity.com/paper/2409.11951/) (2024)

## Built on by (most cited first, 12 of 12)

- [Audio-Driven Universal Gaussian Head Avatars](https://mlsanity.com/paper/2509.18924/) (2025)
- [TexAvatars : Hybrid Texel-3D Representations for Stable Rigging of Photorealistic Gaussian Head Avatars](https://mlsanity.com/paper/2512.21099/) (2025)
- [PhysHead: Simulation-Ready Gaussian Head Avatars](https://mlsanity.com/paper/2604.06467/) (2026)
- [High-Fidelity Mobile Avatars with Pruned Local Blendshapes](https://mlsanity.com/paper/2605.01854/) (2026)
- [Frequency-Decomposed Avatar Representation for Varying Camera Distances](https://mlsanity.com/paper/2512.03593/) (2025)
- [Registration-Free Learnable Multi-View Capture of Faces in Dense Semantic Correspondence](https://mlsanity.com/paper/2605.01450/) (2026)
- [Multi-view Consistent 3D Gaussian Head Avatars 'without' Multi-view Generation](https://mlsanity.com/paper/2605.25220/) (2026)
- [Instant Expressive Gaussian Head Avatars at Over 100 FPS](https://mlsanity.com/paper/2512.16893/) (2025)
- [RelightAnyone: A Generalized Relightable 3D Gaussian Head Model](https://mlsanity.com/paper/2601.03357/) (2026)
- [One-shot Compositional 3D Head Avatars with Deformable Hair](https://mlsanity.com/paper/2604.14782/) (2026)
- [Multi-View Face and Gesture Animation with Dynamic Gaussians](https://mlsanity.com/paper/2608.04722/) (2026)
- [GHARP: Real-time Gaussian Head Animation from Large-scale Reconstruction Prior](https://mlsanity.com/paper/2610.10945/) (2026)

## Abstract (arXiv)

Generating high-fidelity real-time animated sequences of photorealistic 3D head avatars is important for many graphics applications, including immersive telepresence and movies. This is a challenging problem particularly when rendering digital avatar close-ups for showing character's facial microfeatures and expressions. To capture the expressive, detailed nature of human heads, including skin furrowing and finer-scale facial movements, we propose to couple locally-defined facial expressions with 3D Gaussian splatting to enable creating ultra-high fidelity, expressive and photorealistic 3D head avatars. In contrast to previous works that operate on a global expression space, we condition our avatar's dynamics on patch-based local expression features and synthesize 3D Gaussians at a patch level. In particular, we leverage a patch-based geometric 3D face model to extract patch expressions and learn how to translate these into local dynamic skin appearance and motion by coupling the patches with anchor points of Scaffold-GS, a recent hierarchical scene representation. These anchors are then used to synthesize 3D Gaussians on-the-fly, conditioned by patch-expressions and viewing direction. We employ color-based densification and progressive training to obtain high-quality results and faster convergence for high resolution 3K training images. By leveraging patch-level expressions, ScaffoldAvatar consistently achieves state-of-the-art performance with visually natural motion, while encompassing diverse facial expressions and styles in real time.