# Learning a Generalized Physical Face Model From Data

arXiv: https://arxiv.org/abs/2402.19477 · HTML: https://mlsanity.com/paper/2402.19477/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [3D Morphable Face Models -- Past, Present and Future](https://mlsanity.com/paper/1909.01815/) (2019)
- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction](https://mlsanity.com/paper/2003.13989/) (2020)
- [NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads](https://mlsanity.com/paper/2305.03027/) (2023)
- [Towards Metrical Reconstruction of Human Faces](https://mlsanity.com/paper/2204.06607/) (2022)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Learning Smooth Neural Functions via Lipschitz Regularization](https://mlsanity.com/paper/2202.08345/) (2022)
- [ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2203.14510/) (2022)
- [Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videos](https://mlsanity.com/paper/2207.11094/) (2022)
- [Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/2105.07474/) (2021)
- [REALY: Rethinking the Evaluation of 3D Face Reconstruction](https://mlsanity.com/paper/2203.09729/) (2022)
- [Survey on 3D face reconstruction from uncalibrated images](https://mlsanity.com/paper/2011.05740/) (2020)
- [High-Quality Face Capture Using Anatomical Muscles](https://mlsanity.com/paper/1812.02836/) (2018)
- [A Perceptual Shape Loss for Monocular 3D Face Reconstruction](https://mlsanity.com/paper/2310.19580/) (2023)
- [Implicit Neural Representation for Physics-driven Actuated Soft Bodies](https://mlsanity.com/paper/2401.14861/) (2024)
- [SCULPTOR: Skeleton-Consistent Face Creation Using a Learned Parametric Generator](https://mlsanity.com/paper/2209.06423/) (2022)
- [Efficient Incremental Potential Contact for Actuated Face Simulation](https://mlsanity.com/paper/2312.02999/) (2023)

## Built on by (most cited first, 6 of 6)

- [Facing Asymmetry -- Uncovering the Causal Link between Facial Symmetry and Expression Classifiers using Synthetic Interventions](https://mlsanity.com/paper/2409.15927/) (2024)
- [BFSM: 3D Bidirectional Face-Skull Morphable Model](https://mlsanity.com/paper/2509.24577/) (2025)
- [GNM Head: A Generative aNthropometric Model of the human head](https://mlsanity.com/paper/2607.23687/) (2026)
- [Electromyography-Informed Facial Expression Reconstruction for Physiological-Based Synthesis and Analysis](https://mlsanity.com/paper/2503.09556/) (2025)
- [3D Blood Pulsation Maps](https://mlsanity.com/paper/2512.10517/) (2025)
- [SOMA: From Surface Observations to Muscle Anatomy](https://mlsanity.com/paper/2606.09246/) (2026)

## Abstract (arXiv)

Physically-based simulation is a powerful approach for 3D facial animation as the resulting deformations are governed by physical constraints, allowing to easily resolve self-collisions, respond to external forces and perform realistic anatomy edits. Today's methods are data-driven, where the actuations for finite elements are inferred from captured skin geometry. Unfortunately, these approaches have not been widely adopted due to the complexity of initializing the material space and learning the deformation model for each character separately, which often requires a skilled artist followed by lengthy network training. In this work, we aim to make physics-based facial animation more accessible by proposing a generalized physical face model that we learn from a large 3D face dataset. Once trained, our model can be quickly fit to any unseen identity and produce a ready-to-animate physical face model automatically. Fitting is as easy as providing a single 3D face scan, or even a single face image. After fitting, we offer intuitive animation controls, as well as the ability to retarget animations across characters. All the while, the resulting animations allow for physical effects like collision avoidance, gravity, paralysis, bone reshaping and more.