# High-Quality Real Time Facial Capture Based on Single Camera

arXiv: https://arxiv.org/abs/2111.07556 · HTML: https://mlsanity.com/paper/2111.07556/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 37 references that are in the map.
- First posted on arXiv 15 Nov 2021.

## Builds on (most cited first, 4 of 4)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Production-Level Facial Performance Capture Using Deep Convolutional Neural Networks](https://mlsanity.com/paper/1609.06536/) (2016)
- [LSTM-Based Facial Performance Capture Using Embedding Between Expressions](https://mlsanity.com/paper/1805.03874/) (2018)

## Built on by (most cited first, 2 of 2)

- [EmoDiffusion: Enhancing Emotional 3D Facial Animation with Latent Diffusion Models](https://mlsanity.com/paper/2503.11028/) (2025)
- [4D Agnostic Real-Time Facial Animation Pipeline for Desktop Scenarios](https://mlsanity.com/paper/2304.02814/) (2023)

## Abstract (arXiv)

We propose a real time deep learning framework for video-based facial expression capture. Our process uses a high-end facial capture pipeline based on FACEGOOD to capture facial expression. We train a convolutional neural network to produce high-quality continuous blendshape weight output from video training. Since this facial capture is fully automated, our system can drastically reduce the amount of labor involved in the development of modern narrative-driven video games or films involving realistic digital doubles of actors and potentially hours of animated dialogue per character. We demonstrate compelling animation inference in challenging areas such as eyes and lips.