# Monocular and Generalizable Gaussian Talking Head Animation

arXiv: https://arxiv.org/abs/2504.00665 · HTML: https://mlsanity.com/paper/2504.00665/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 70 references that are in the map.
- First posted on arXiv 1 Apr 2025.

## Builds on (most cited first, 30 of 52)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation](https://mlsanity.com/paper/2211.12194/) (2022)
- [AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis](https://mlsanity.com/paper/2103.11078/) (2021)
- [MakeItTalk: Speaker-Aware Talking-Head Animation](https://mlsanity.com/paper/2004.12992/) (2020)
- [GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image](https://mlsanity.com/paper/2403.12013/) (2024)
- [GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians](https://mlsanity.com/paper/2312.02069/) (2023)
- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [Sapiens: Foundation for Human Vision Models](https://mlsanity.com/paper/2408.12569/) (2024)
- [PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering](https://mlsanity.com/paper/2109.08379/) (2021)
- [EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions](https://mlsanity.com/paper/2407.08136/) (2024)
- [NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads](https://mlsanity.com/paper/2305.03027/) (2023)
- [Depth-Aware Generative Adversarial Network for Talking Head Video Generation](https://mlsanity.com/paper/2203.06605/) (2022)
- [SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting](https://mlsanity.com/paper/2403.05087/) (2024)
- [GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis](https://mlsanity.com/paper/2312.02155/) (2023)
- [StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN](https://mlsanity.com/paper/2203.04036/) (2022)
- [Lip Movements Generation at a Glance](https://mlsanity.com/paper/1803.10404/) (2018)
- [Realistic One-shot Mesh-based Head Avatars](https://mlsanity.com/paper/2206.08343/) (2022)
- [Talking-head Generation with Rhythmic Head Motion](https://mlsanity.com/paper/2007.08547/) (2020)
- [Identity-Preserving Talking Face Generation with Landmark and Appearance Priors](https://mlsanity.com/paper/2305.08293/) (2023)
- [Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis](https://mlsanity.com/paper/2401.08503/) (2024)
- [Towards Automatic Face-to-Face Translation](https://mlsanity.com/paper/2003.00418/) (2020)
- [StyleTalk: One-shot Talking Head Generation with Controllable Speaking Styles](https://mlsanity.com/paper/2301.01081/) (2023)
- [One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning](https://mlsanity.com/paper/2112.02749/) (2021)
- [GPAvatar: Generalizable and Precise Head Avatar from Image(s)](https://mlsanity.com/paper/2401.10215/) (2024)

## Built on by (most cited first, 10 of 10)

- [Beyond Isolated Words: Diffusion Brush for Handwritten Text-Line Generation](https://mlsanity.com/paper/2508.03256/) (2025)
- [DyStream: Streaming Dyadic Talking Heads Generation via Flow Matching-based Autoregressive Model](https://mlsanity.com/paper/2512.24408/) (2025)
- [FastGHA: Generalized Few-Shot 3D Gaussian Head Avatars with Real-Time Animation](https://mlsanity.com/paper/2601.13837/) (2026)
- [FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editing](https://mlsanity.com/paper/2505.22141/) (2025)
- [EmoDiffTalk:Emotion-aware Diffusion for Editable 3D Gaussian Talking Head](https://mlsanity.com/paper/2512.05991/) (2025)
- [Toward Fine-Grained Facial Control in 3D Talking Head Generation](https://mlsanity.com/paper/2602.09736/) (2026)
- [VDE: Training-Free Accelerating Rectified Flow Model via Velocity Decomposition and Estimation](https://mlsanity.com/paper/2605.23381/) (2026)
- [HM-Talker: Hybrid Motion Modeling for High-Fidelity Talking Head Synthesis](https://mlsanity.com/paper/2508.10566/) (2025)
- [Learning Hierarchical and Geometry-Aware Graph Representations for Text-to-CAD](https://mlsanity.com/paper/2604.10075/) (2026)
- [EmoZone-Talker: Regional Semantic Control of Audio-Driven 3DGS Talking Heads via Facial Action Units](https://mlsanity.com/paper/2606.15848/) (2026)

## Abstract (arXiv)

In this work, we introduce Monocular and Generalizable Gaussian Talking Head Animation (MGGTalk), which requires monocular datasets and generalizes to unseen identities without personalized re-training. Compared with previous 3D Gaussian Splatting (3DGS) methods that requires elusive multi-view datasets or tedious personalized learning/inference, MGGtalk enables more practical and broader applications. However, in the absence of multi-view and personalized training data, the incompleteness of geometric and appearance information poses a significant challenge. To address these challenges, MGGTalk explores depth information to enhance geometric and facial symmetry characteristics to supplement both geometric and appearance features. Initially, based on the pixel-wise geometric information obtained from depth estimation, we incorporate symmetry operations and point cloud filtering techniques to ensure a complete and precise position parameter for 3DGS. Subsequently, we adopt a two-stage strategy with symmetric priors for predicting the remaining 3DGS parameters. We begin by predicting Gaussian parameters for the visible facial regions of the source image. These parameters are subsequently utilized to improve the prediction of Gaussian parameters for the non-visible regions. Extensive experiments demonstrate that MGGTalk surpasses previous state-of-the-art methods, achieving superior performance across various metrics.