# Sparse to Dense Dynamic 3D Facial Expression Generation

arXiv: https://arxiv.org/abs/2105.07463 · HTML: https://mlsanity.com/paper/2105.07463/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 16 May 2021.

## Builds on (most cited first, 24 of 24)

- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [Capture, Learning, and Synthesis of 3D Speaking Styles](https://mlsanity.com/paper/1905.03079/) (2019)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018)
- [Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation](https://mlsanity.com/paper/1905.02876/) (2019)
- [Deformable Shape Completion with Graph Convolutional Autoencoders](https://mlsanity.com/paper/1712.00268/) (2017)
- [Automatic Face Reenactment](https://mlsanity.com/paper/1602.02651/) (2016)
- [A Simple Approach to Intrinsic Correspondence Learning on Unstructured 3D Meshes](https://mlsanity.com/paper/1809.06664/) (2018)
- [Multilinear Wavelets: A Statistical Shape Space for Human Faces](https://mlsanity.com/paper/1401.2818/) (2014)
- [Gaussian Process Morphable Models](https://mlsanity.com/paper/1603.07254/) (2016)
- [Supervision by Registration and Triangulation for Landmark Detection](https://mlsanity.com/paper/2101.09866/) (2021)
- [ExpNet: Landmark-Free, Deep, 3D Facial Expressions](https://mlsanity.com/paper/1802.00542/) (2018)
- [3DFaceGAN: Adversarial Nets for 3D Face Representation, Generation, and Translation](https://mlsanity.com/paper/1905.00307/) (2019)
- [Every Smile is Unique: Landmark-Guided Diverse Smile Generation](https://mlsanity.com/paper/1802.01873/) (2018)
- [FacePoseNet: Making a Case for Landmark-Free Face Alignment](https://mlsanity.com/paper/1708.07517/) (2017)
- [Dynamic Facial Expression Generation on Hilbert Hypersphere with Conditional Wasserstein Generative Adversarial Nets](https://mlsanity.com/paper/1907.10087/) (2019)
- [Robust Facial Landmark Detection by Multi-order Multi-constraint Deep Networks](https://mlsanity.com/paper/2012.04927/) (2020)
- [Controllable Image-to-Video Translation: A Case Study on Facial Expression Generation](https://mlsanity.com/paper/1808.02992/) (2018)
- [A Sparse and Locally Coherent Morphable Face Model for Dense Semantic Correspondence Across Heterogeneous 3D Faces](https://mlsanity.com/paper/2006.03840/) (2020)
- [Learning to Generate Customized Dynamic 3D Facial Expressions](https://mlsanity.com/paper/2007.09805/) (2020)
- [A Novel Space-Time Representation on the Positive Semidefinite Con for Facial Expression Recognition](https://mlsanity.com/paper/1707.06440/) (2017)
- [Deep Structured Prediction for Facial Landmark Detection](https://mlsanity.com/paper/2010.09035/) (2020)

## Built on by (most cited first, 19 of 19)

- [MMHead: Towards Fine-grained Multi-modal 3D Facial Animation](https://mlsanity.com/paper/2410.07757/) (2024)
- [4D Facial Expression Diffusion Model](https://mlsanity.com/paper/2303.16611/) (2023)
- [EmoVOCA: Speech-Driven Emotional 3D Talking Heads](https://mlsanity.com/paper/2403.12886/) (2024)
- [Reversible Graph Neural Network-based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation](https://mlsanity.com/paper/2305.15270/) (2023)
- [Learning Landmarks Motion from Speech for Speaker-Agnostic 3D Talking Heads Generation](https://mlsanity.com/paper/2306.01415/) (2023)
- [The Florence 4D Facial Expression Dataset](https://mlsanity.com/paper/2210.16807/) (2022)
- [Generating Multiple 4D Expression Transitions by Learning Face Landmark Trajectories](https://mlsanity.com/paper/2208.00050/) (2022)
- [3D Generative Model Latent Disentanglement via Local Eigenprojection](https://mlsanity.com/paper/2302.12798/) (2023)
- [Toward Mesh-Invariant 3D Generative Deep Learning with Geometric Measures](https://mlsanity.com/paper/2306.15762/) (2023)
- [AnimateMe: 4D Facial Expressions via Diffusion Models](https://mlsanity.com/paper/2403.17213/) (2024)
- [Basis restricted elastic shape analysis on the space of unregistered surfaces](https://mlsanity.com/paper/2311.04382/) (2023)
- [Beyond Fixed Topologies: Unregistered Training and Comprehensive Evaluation Metrics for 3D Talking Heads](https://mlsanity.com/paper/2410.11041/) (2024)
- [JambaTalk: Speech-Driven 3D Talking Head Generation Based on Hybrid Transformer-Mamba Model](https://mlsanity.com/paper/2408.01627/) (2024)
- [LSF-Animation: Label-Free Speech-Driven Facial Animation via Implicit Feature Representation](https://mlsanity.com/paper/2510.21864/) (2025)
- [End-to-end Face-swapping via Adaptive Latent Representation Learning](https://mlsanity.com/paper/2303.04186/) (2023)
- [Advances in 4D Generation: A Survey](https://mlsanity.com/paper/2503.14501/) (2025)
- [Instruction-Driven 3D Facial Expression Generation and Transition](https://mlsanity.com/paper/2601.08179/) (2026)
- [FreeTalk: Emotional Topology-Free 3D Talking Heads](https://mlsanity.com/paper/2603.15512/) (2026)
- [Cepheid: Decoupling Scale from Structure for Cross-Family Motion Completion](https://mlsanity.com/paper/2610.18871/) (2026)

## Abstract (arXiv)

In this paper, we propose a solution to the task of generating dynamic 3D facial expressions from a neutral 3D face and an expression label. This involves solving two sub-problems: (i)modeling the temporal dynamics of expressions, and (ii) deforming the neutral mesh to obtain the expressive counterpart. We represent the temporal evolution of expressions using the motion of a sparse set of 3D landmarks that we learn to generate by training a manifold-valued GAN (Motion3DGAN). To better encode the expression-induced deformation and disentangle it from the identity information, the generated motion is represented as per-frame displacement from a neutral configuration. To generate the expressive meshes, we train a Sparse2Dense mesh Decoder (S2D-Dec) that maps the landmark displacements to a dense, per-vertex displacement. This allows us to learn how the motion of a sparse set of landmarks influences the deformation of the overall face surface, independently from the identity. Experimental results on the CoMA and D3DFACS datasets show that our solution brings significant improvements with respect to previous solutions in terms of both dynamic expression generation and mesh reconstruction, while retaining good generalization to unseen data. The code and the pretrained model will be made publicly available.