# Real-time 3D Facial Tracking via Cascaded Compositional Learning

arXiv: https://arxiv.org/abs/2009.00935 · HTML: https://mlsanity.com/paper/2009.00935/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Sep 2020.

## Built on by (most cited first, 4 of 4)

- [Partition Speeds Up Learning Implicit Neural Representations Based on Exponential-Increase Hypothesis](https://mlsanity.com/paper/2310.14184/) (2023)
- [MagicPortrait: Temporally Consistent Face Reenactment with 3D Geometric Guidance](https://mlsanity.com/paper/2504.21497/) (2025)
- [HeadRecon: High-Fidelity 3D Head Reconstruction from Monocular Video](https://mlsanity.com/paper/2312.08863/) (2023)
- [SelfieAvatar: Real-time Head Avatar reenactment from a Selfie Video](https://mlsanity.com/paper/2601.18851/) (2026)

## Abstract (arXiv)

We propose to learn a cascade of globally-optimized modular boosted ferns (GoMBF) to solve multi-modal facial motion regression for real-time 3D facial tracking from a monocular RGB camera. GoMBF is a deep composition of multiple regression models with each is a boosted ferns initially trained to predict partial motion parameters of the same modality, and then concatenated together via a global optimization step to form a singular strong boosted ferns that can effectively handle the whole regression target. It can explicitly cope with the modality variety in output variables, while manifesting increased fitting power and a faster learning speed comparing against the conventional boosted ferns. By further cascading a sequence of GoMBFs (GoMBF-Cascade) to regress facial motion parameters, we achieve competitive tracking performance on a variety of in-the-wild videos comparing to the state-of-the-art methods, which require much more training data or have higher computational complexity. It provides a robust and highly elegant solution to real-time 3D facial tracking using a small set of training data and hence makes it more practical in real-world applications.