# Learning to Generate Customized Dynamic 3D Facial Expressions

arXiv: https://arxiv.org/abs/2007.09805 · HTML: https://mlsanity.com/paper/2007.09805/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jul 2020.

## Built on by (most cited first, 11 of 11)

- [Sparse to Dense Dynamic 3D Facial Expression Generation](https://mlsanity.com/paper/2105.07463/) (2021)
- [4D Facial Expression Diffusion Model](https://mlsanity.com/paper/2303.16611/) (2023)
- [Revisiting Point Cloud Simplification: A Learnable Feature Preserving Approach](https://mlsanity.com/paper/2109.14982/) (2021)
- [GraphWalks: Efficient Shape Agnostic Geodesic Shortest Path Estimation](https://mlsanity.com/paper/2205.15217/) (2022)
- [The Florence 4D Facial Expression Dataset](https://mlsanity.com/paper/2210.16807/) (2022)
- [Generating Multiple 4D Expression Transitions by Learning Face Landmark Trajectories](https://mlsanity.com/paper/2208.00050/) (2022)
- [ShapeFusion: A 3D diffusion model for localized shape editing](https://mlsanity.com/paper/2403.19773/) (2024)
- [AnimateMe: 4D Facial Expressions via Diffusion Models](https://mlsanity.com/paper/2403.17213/) (2024)
- [3D human tongue reconstruction from single "in-the-wild" images](https://mlsanity.com/paper/2106.12302/) (2021)
- [ImHead: A Large-scale Implicit Morphable Model for Localized Head Modeling](https://mlsanity.com/paper/2510.10793/) (2025)
- [A Review of Affective Generation Models](https://mlsanity.com/paper/2202.10763/) (2022)

## Abstract (arXiv)

Recent advances in deep learning have significantly pushed the state-of-the-art in photorealistic video animation given a single image. In this paper, we extrapolate those advances to the 3D domain, by studying 3D image-to-video translation with a particular focus on 4D facial expressions. Although 3D facial generative models have been widely explored during the past years, 4D animation remains relatively unexplored. To this end, in this study we employ a deep mesh encoder-decoder like architecture to synthesize realistic high resolution facial expressions by using a single neutral frame along with an expression identification. In addition, processing 3D meshes remains a non-trivial task compared to data that live on grid-like structures, such as images. Given the recent progress in mesh processing with graph convolutions, we make use of a recently introduced learnable operator which acts directly on the mesh structure by taking advantage of local vertex orderings. In order to generalize to 4D facial expressions across subjects, we trained our model using a high resolution dataset with 4D scans of six facial expressions from 180 subjects. Experimental results demonstrate that our approach preserves the subject's identity information even for unseen subjects and generates high quality expressions. To the best of our knowledge, this is the first study tackling the problem of 4D facial expression synthesis.