# Dense 3D Face Decoding over 2500FPS: Joint Texture & Shape Convolutional Mesh Decoders

arXiv: https://arxiv.org/abs/1904.03525 · HTML: https://mlsanity.com/paper/1904.03525/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2019.

## Built on by (most cited first, 22 of 22)

- [Pixel Codec Avatars](https://mlsanity.com/paper/2104.04638/) (2021)
- [3D face reconstruction with dense landmarks](https://mlsanity.com/paper/2204.02776/) (2022)
- [AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs](https://mlsanity.com/paper/2112.05957/) (2021)
- [Normalized Avatar Synthesis Using StyleGAN and Perceptual Refinement](https://mlsanity.com/paper/2106.11423/) (2021)
- [FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction](https://mlsanity.com/paper/2111.01082/) (2021)
- [SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction](https://mlsanity.com/paper/2106.03021/) (2021)
- [Self-Supervised 3D Face Reconstruction via Conditional Estimation](https://mlsanity.com/paper/2110.04800/) (2021)
- [MeInGame: Create a Game Character Face from a Single Portrait](https://mlsanity.com/paper/2102.02371/) (2021)
- [Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection](https://mlsanity.com/paper/2106.07852/) (2021)
- [Learning Free-Form Deformation for 3D Face Reconstruction from In-The-Wild Images](https://mlsanity.com/paper/2105.14857/) (2021)
- [LUCAS: Layered Universal Codec Avatars](https://mlsanity.com/paper/2502.19739/) (2025)
- [RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairs](https://mlsanity.com/paper/2302.05486/) (2023)
- [Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape](https://mlsanity.com/paper/2204.04379/) (2022)
- [Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression](https://mlsanity.com/paper/2208.07142/) (2022)
- [ImHead: A Large-scale Implicit Morphable Model for Localized Head Modeling](https://mlsanity.com/paper/2510.10793/) (2025)
- [Graphics Capsule: Learning Hierarchical 3D Face Representations from 2D Images](https://mlsanity.com/paper/2303.10896/) (2023)
- [High-Quality 3D Face Reconstruction with Affine Convolutional Networks](https://mlsanity.com/paper/2310.14237/) (2023)
- [3D Dense Face Alignment with Fused Features by Aggregating CNNs and GCNs](https://mlsanity.com/paper/2203.04643/) (2022)
- [A Review of 3D Face Reconstruction From a Single Image](https://mlsanity.com/paper/2110.09299/) (2021)
- [BabyNet: Reconstructing 3D faces of babies from uncalibrated photographs](https://mlsanity.com/paper/2203.05908/) (2022)
- [MUNCH: Modelling Unique 'N Controllable Heads](https://mlsanity.com/paper/2310.02753/) (2023)
- [Deep Spectral Meshes: Multi-Frequency Facial Mesh Processing with Graph Neural Networks](https://mlsanity.com/paper/2402.10365/) (2024)

## Abstract (arXiv)

3D Morphable Models (3DMMs) are statistical models that represent facial texture and shape variations using a set of linear bases and more particular Principal Component Analysis (PCA). 3DMMs were used as statistical priors for reconstructing 3D faces from images by solving non-linear least square optimization problems. Recently, 3DMMs were used as generative models for training non-linear mappings (ie, regressors) from image to the parameters of the models via Deep Convolutional Neural Networks (DCNNs). Nevertheless, all of the above methods use either fully connected layers or 2D convolutions on parametric unwrapped UV spaces leading to large networks with many parameters. In this paper, we present the first, to the best of our knowledge, non-linear 3DMMs by learning joint texture and shape auto-encoders using direct mesh convolutions. We demonstrate how these auto-encoders can be used to train very light-weight models that perform Coloured Mesh Decoding (CMD) in-the-wild at a speed of over 2500 FPS.