# GenSync: A Generalized Talking Head Framework for Audio-driven Multi-Subject Lip-Sync using 3D Gaussian Splatting

arXiv: https://arxiv.org/abs/2505.01928 · HTML: https://mlsanity.com/paper/2505.01928/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 3 May 2025.

## Builds on (most cited first, 15 of 15)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis](https://mlsanity.com/paper/2103.11078/) (2021)
- [3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting](https://mlsanity.com/paper/2312.09228/) (2023)
- [X2Face: A network for controlling face generation by using images, audio, and pose codes](https://mlsanity.com/paper/1807.10550/) (2018)
- [DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models](https://mlsanity.com/paper/2304.00916/) (2023)
- [Morphable Face Models - An Open Framework](https://mlsanity.com/paper/1709.08398/) (2017)
- [X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attention](https://mlsanity.com/paper/2403.15931/) (2024)
- [SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis](https://mlsanity.com/paper/2311.17590/) (2023)
- [Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization](https://mlsanity.com/paper/2308.09716/) (2023)
- [Masked Lip-Sync Prediction by Audio-Visual Contextual Exploitation in Transformers](https://mlsanity.com/paper/2212.04970/) (2022)
- [TALK-Act: Enhance Textural-Awareness for 2D Speaking Avatar Reenactment with Diffusion Model](https://mlsanity.com/paper/2410.10696/) (2024)
- [SyncNet: correlating objective for time delay estimation in audio signals](https://mlsanity.com/paper/2203.14639/) (2022)
- [MI-NeRF: Learning a Single Face NeRF from Multiple Identities](https://mlsanity.com/paper/2403.19920/) (2024)

## Built on by (most cited first, 1 of 1)

- [FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editing](https://mlsanity.com/paper/2505.22141/) (2025)

## Abstract (arXiv)

We introduce GenSync, a novel framework for multi-identity lip-synced video synthesis using 3D Gaussian Splatting. Unlike most existing 3D methods that require training a new model for each identity , GenSync learns a unified network that synthesizes lip-synced videos for multiple speakers. By incorporating a Disentanglement Module, our approach separates identity-specific features from audio representations, enabling efficient multi-identity video synthesis. This design reduces computational overhead and achieves 6.8x faster training compared to state-of-the-art models, while maintaining high lip-sync accuracy and visual quality.