An Implicit Physical Face Model Driven by Expression and Style
Not on the map. Why?
Abstract (arXiv)
3D facial animation is often produced by manipulating facial deformation models (or rigs), that are traditionally parameterized by expression controls. A key component that is usually overlooked is expression 'style', as in, how a particular expression is performed. Although it is common to define a semantic basis of expressions that characters can perform, most characters perform each expression in their own style. To date, style is usually entangled with the expression, and it is not possible to transfer the style of one character to another when considering facial animation. We present a new face model, based on a data-driven implicit neural physics model, that can be driven by both expression and style separately. At the core, we present a framework for learning implicit physics-based actuations for multiple subjects simultaneously, trained on a few arbitrary performance capture sequences from a small set of identities. Once trained, our method allows generalized physics-based facial animation for any of the trained identities, extending to unseen performances. Furthermore, it grants control over the animation style, enabling style transfer from one character to another or blending styles of different characters. Lastly, as a physics-based model, it is capable of synthesizing physical effects, such as collision handling, setting our method apart from conventional approaches.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 21 of its 59 references that are in the map. First posted on arXiv 27 Jan 2024.
Builds on
21 of 59 references.
References · 21
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
- 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
- 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 312
- 20193D Morphable Face Models -- Past, Present and Futurecited by 270
- 2019Capture, Learning, and Synthesis of 3D Speaking Stylescited by 259
- 2017Neural Style Transfer: A Reviewcited by 199
- 2022RigNeRF: Fully Controllable Neural 3D Portraitscited by 126
- 2020Unpaired Motion Style Transfer from Video to Animationcited by 94
- 2022Learning Smooth Neural Functions via Lipschitz Regularizationcited by 86
- 2019Dynamic Instance Normalization for Arbitrary Style Transfercited by 83
- 2022Imitator: Personalized Speech-driven 3D Facial Animationcited by 74
9 more
- 2021LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalizationcited by 67
- 2022Motion Puzzle: Arbitrary Motion Style Transfer by Body Partcited by 64
- 2021DiffPD: Differentiable Projective Dynamicscited by 62
- 2021Riggable 3D Face Reconstruction via In-Network Optimizationcited by 31
- 2019Joint Face Detection and Facial Motion Retargeting for Multiple Facescited by 18
- 2020Facial Expression Retargeting from Human to Avatar Made Easycited by 13
- 2018High-Quality Face Capture Using Anatomical Musclescited by 8
- 2024Implicit Neural Representation for Physics-driven Actuated Soft Bodiescited by 5
- 2023Style Transfer for 2D Talking Head Animationcited by 3
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
- 2026GNM Head: A Generative aNthropometric Model of the human head26 Jul 2026
- 2025BFSM: 3D Bidirectional Face-Skull Morphable Model29 Sep 2025
- 2025Improving Facial Rig Semantics for Tracking and Retargeting11 Aug 2025