# Geometry-guided Dense Perspective Network for Speech-Driven Facial Animation

arXiv: https://arxiv.org/abs/2008.10004 · HTML: https://mlsanity.com/paper/2008.10004/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2020.

## Built on by (most cited first, 19 of 19)

- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [FaceFormer: Speech-Driven 3D Facial Animation with Transformers](https://mlsanity.com/paper/2112.05329/) (2021)
- [VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction](https://mlsanity.com/paper/2501.01957/) (2025)
- [EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation](https://mlsanity.com/paper/2303.11089/) (2023)
- [Deep Learning for Visual Speech Analysis: A Survey](https://mlsanity.com/paper/2205.10839/) (2022)
- [Learn2Talk: 3D Talking Face Learns from 2D Talking Face](https://mlsanity.com/paper/2404.12888/) (2024)
- [Learning Audio-Driven Viseme Dynamics for 3D Face Animation](https://mlsanity.com/paper/2301.06059/) (2023)
- [MemoryTalker: Personalized Speech-Driven 3D Facial Animation via Audio-Guided Stylization](https://mlsanity.com/paper/2507.20562/) (2025)
- [MMFace4D: A Large-Scale Multi-Modal 4D Face Dataset for Audio-Driven 3D Face Animation](https://mlsanity.com/paper/2303.09797/) (2023)
- [CorrTalk: Correlation Between Hierarchical Speech and Facial Activity Variances for 3D Animation](https://mlsanity.com/paper/2310.11295/) (2023)
- [OT-Talk: Animating 3D Talking Head with Optimal Transportation](https://mlsanity.com/paper/2505.01932/) (2025)
- [Expressive Speech-driven Facial Animation with controllable emotions](https://mlsanity.com/paper/2301.02008/) (2023)
- [Breathing Life into Faces: Speech-driven 3D Facial Animation with Natural Head Pose and Detailed Shape](https://mlsanity.com/paper/2310.20240/) (2023)
- [Diverse Code Query Learning for Speech-Driven Facial Animation](https://mlsanity.com/paper/2409.19143/) (2024)
- [FMReward: Aligning and Evaluating Audio-Driven 3D Facial Animation with Human Preferences](https://mlsanity.com/paper/2608.15296/) (2026)
- [Pose-Controllable 3D Facial Animation Synthesis using Hierarchical Audio-Vertex Attention](https://mlsanity.com/paper/2302.12532/) (2023)
- [Towards Unified Co-Speech Gesture Generation via Hierarchical Implicit Periodicity Learning](https://mlsanity.com/paper/2512.13131/) (2025)
- [MMTalker: Multiresolution 3D Talking Head Synthesis with Multimodal Feature Fusion](https://mlsanity.com/paper/2604.02941/) (2026)
- [AnyTalk: Speech Animation for Arbitrary Characters Leveraging a Video Generation Model](https://mlsanity.com/paper/2608.16143/) (2026)

## Abstract (arXiv)

Realistic speech-driven 3D facial animation is a challenging problem due to the complex relationship between speech and face. In this paper, we propose a deep architecture, called Geometry-guided Dense Perspective Network (GDPnet), to achieve speaker-independent realistic 3D facial animation. The encoder is designed with dense connections to strengthen feature propagation and encourage the re-use of audio features, and the decoder is integrated with an attention mechanism to adaptively recalibrate point-wise feature responses by explicitly modeling interdependencies between different neuron units. We also introduce a non-linear face reconstruction representation as a guidance of latent space to obtain more accurate deformation, which helps solve the geometry-related deformation and is good for generalization across subjects. Huber and HSIC (Hilbert-Schmidt Independence Criterion) constraints are adopted to promote the robustness of our model and to better exploit the non-linear and high-order correlations. Experimental results on the public dataset and real scanned dataset validate the superiority of our proposed GDPnet compared with state-of-the-art model.