# Super-realtime facial landmark detection and shape fitting by deep regression of shape model parameters

arXiv: https://arxiv.org/abs/1902.03459 · HTML: https://mlsanity.com/paper/1902.03459/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2019.

## Built on by (most cited first, 2 of 2)

- [Learning a Structured Latent Space for Unsupervised Point Cloud Completion](https://mlsanity.com/paper/2203.15580/) (2022)
- [FSFSplatter: Build Surface and Novel Views with Sparse-Views within 2min](https://mlsanity.com/paper/2510.02691/) (2025)

## Abstract (arXiv)

We present a method for highly efficient landmark detection that combines deep convolutional neural networks with well established model-based fitting algorithms. Motivated by established model-based fitting methods such as active shapes, we use a PCA of the landmark positions to allow generative modeling of facial landmarks. Instead of computing the model parameters using iterative optimization, the PCA is included in a deep neural network using a novel layer type. The network predicts model parameters in a single forward pass, thereby allowing facial landmark detection at several hundreds of frames per second. Our architecture allows direct end-to-end training of a model-based landmark detection method and shows that deep neural networks can be used to reliably predict model parameters directly without the need for an iterative optimization. The method is evaluated on different datasets for facial landmark detection and medical image segmentation. PyTorch code is freely available at https://github.com/justusschock/shapenet