mlsanityData to 7 Oct 2026

GaussianHeadTalk: Wobble-Free 3D Talking Heads with Audio Driven Gaussian Splatting

arXiv 2512.10939 ↗First posted 11 Dec 2025cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Speech-driven talking heads have recently emerged and enable interactive avatars. However, real-world applications are limited, as current methods achieve high visual fidelity but slow or fast yet temporally unstable. Diffusion methods provide realistic image generation, yet struggle with oneshot settings. Gaussian Splatting approaches are real-time, yet inaccuracies in facial tracking, or inconsistent Gaussian mappings, lead to unstable outputs and video artifacts that are detrimental to realistic use cases. We address this problem by mapping Gaussian Splatting using 3D Morphable Models to generate person-specific avatars. We introduce transformer-based prediction of model parameters, directly from audio, to drive temporal consistency. From monocular video and independent audio speech inputs, our method enables generation of real-time talking head videos where we report competitive quantitative and qualitative performance.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 50 of its 69 references that are in the map. First posted on arXiv 11 Dec 2025.

Builds on

50 of 69 references.

References · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  7. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  8. 2020First Order Motion Model for Image Animationcited by 698
  9. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 669
  10. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  11. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  12. 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 341
38 more
  1. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  2. 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 310
  3. 2019Capture, Learning, and Synthesis of 3D Speaking Stylescited by 257
  4. 2019Neural Voice Puppetry: Audio-driven Facial Reenactmentcited by 249
  5. 2021FaceFormer: Speech-Driven 3D Facial Animation with Transformerscited by 208
  6. 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
  7. 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 198
  8. 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
  9. 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 191
  10. 2021MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglementcited by 189
  11. 2023CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Priorcited by 183
  12. 2021Audio-Driven Emotional Video Portraitscited by 180
  13. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  14. 2022Instant Volumetric Head Avatarscited by 157
  15. 2022Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generationcited by 119
  16. 2022Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesiscited by 97
  17. 2023Identity-Preserving Talking Face Generation with Landmark and Appearance Priorscited by 97
  18. 2024Generalizable and Animatable Gaussian Head Avatarcited by 95
  19. 2024EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesiscited by 79
  20. 2022Imitator: Personalized Speech-driven 3D Facial Animationcited by 73
  21. 2023SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Facescited by 67
  22. 2024EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatarscited by 65
  23. 2024Portrait4D-v2: Pseudo Multi-View Data Creates Better 4D Head Synthesizercited by 64
  24. 2024TalkingGaussian: Structure-Persistent 3D Talking Head Synthesis via Gaussian Splattingcited by 60
  25. 2023Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Datacited by 55
  26. 2023GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivationcited by 46
  27. 2023STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detectioncited by 41
  28. 2024Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesiscited by 41
  29. 2024GaussianTalker: Speaker-specific Talking Head Synthesis via 3D Gaussian Splattingcited by 24
  30. 2023Generative Face Video Coding Techniques and Standardization Efforts: A Reviewcited by 19
  31. 2022Audio-Visual Face Reenactmentcited by 18
  32. 2024GaussianTalker: Real-Time High-Fidelity Talking Head Synthesis with Audio-Driven 3D Gaussian Splattingcited by 17
  33. 2024MimicTalk: Mimicking a personalized and expressive 3D talking face in minutescited by 16
  34. 2022Compressing Video Calls using Synthetic Talking Headscited by 10
  35. 2024MoDiTalker: Motion-Disentangled Diffusion Model for High-Fidelity Talking Head Generationcited by 10
  36. 2024GaussianSpeech: Audio-Driven Gaussian Avatarscited by 9
  37. 2024Neural Rendering and Its Hardware Acceleration: A Reviewcited by 4
  38. 2024Unveiling the Impact of Multi-Modal Interactions on User Engagement: A Comprehensive Evaluation in AI-driven Conversationscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026EmbedTalk: Talking Head Synthesis using Gaussian Embeddingscited by 0
  2. 2026CETalk: Continuous Valence-Arousal Control for Audio-Driven 3D Talking Head Generationcited by 0