mlsanityData to 14 Oct 2026

Audio2Face-3D: Audio-driven Realistic Facial Animation For Digital Avatars

arXiv 2508.16401 ↗First posted 22 Aug 2025cs.GR cs.HC cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Audio-driven facial animation presents an effective solution for animating digital avatars. In this paper, we detail the technical aspects of NVIDIA Audio2Face-3D, including data acquisition, network architecture, retargeting methodology, evaluation metrics, and use cases. Audio2Face-3D system enables real-time interaction between human users and interactive avatars, facilitating facial animation authoring for game characters. To assist digital avatar creators and game developers in generating realistic facial animations, we have open-sourced Audio2Face-3D networks, SDK, training framework, and example dataset.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 35 of its 59 references that are in the map. First posted on arXiv 22 Aug 2025.

Builds on

35 of 59 references.

References · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  10. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  11. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  12. 2024Mixtral of Expertscited by 1,823
23 more
  1. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
  2. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  3. 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 312
  4. 2019Capture, Learning, and Synthesis of 3D Speaking Stylescited by 259
  5. 2021FaceFormer: Speech-Driven 3D Facial Animation with Transformerscited by 209
  6. 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
  7. 2021MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglementcited by 190
  8. 2021Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motioncited by 149
  9. 2023DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animationcited by 136
  10. 2023GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesiscited by 131
  11. 2023StyleTalk: One-shot Talking Head Generation with Controllable Speaking Stylescited by 92
  12. 2023DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Modelscited by 81
  13. 2023FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusioncited by 78
  14. 2024AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encodingcited by 67
  15. 2023FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Modelscited by 55
  16. 2023TalkCLIP: Talking Head Generation with Text-Guided Expressive Speaking Stylescited by 32
  17. 2023FaceXHuBERT: Text-less Speech-driven E(X)pressive 3D Facial Animation Synthesis Using Self-Supervised Speech Representation Learningcited by 29
  18. 2021Phone-to-audio alignment without text: A Semi-supervised Approachcited by 26
  19. 2024MMHead: Towards Fine-grained Multi-modal 3D Facial Animationcited by 16
  20. 2023ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignmentcited by 13
  21. 2025Voice Cloning: Comprehensive Surveycited by 4
  22. 2024Refined Inverse Rigging: A Balanced Approach to High-fidelity Blendshape Animationcited by 2
  23. 2025Generative Adversarial Network based Voice Conversion: Techniques, Challenges, and Recent Advancementscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2026Ex-Omni: Enabling 3D Facial Animation Generation for Omni-modal Large Language Modelscited by 2
  2. 2025KeyframeFace: Language-Driven Facial Animation via Semantic Keyframescited by 0
  3. 2026SingingBot: An Avatar-Driven System for Robotic Face Singing Performancecited by 0
  4. 2026AudioFace: Language-Assisted Speech-Driven Facial Animation with Multimodal Language Modelscited by 0
  5. 2026Voice "Cloning" is Style Transfercited by 0
  6. 2026Toward Accessible Psychotherapy Training Using AI-Driven Interactive Patient Avatarscited by 0
  7. 2026Mind Companion: An Embodied Conversational Agent for Process-Based Psychotherapycited by 0