# Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression

arXiv: https://arxiv.org/abs/1703.07834 · HTML: https://mlsanity.com/paper/1703.07834/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2017.

## Built on by (most cited first, 30 of 66)

- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [Towards Metrical Reconstruction of Human Faces](https://mlsanity.com/paper/2204.06607/) (2022)
- [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022)
- [Collaborative Regression of Expressive Bodies using Moderation](https://mlsanity.com/paper/2105.05301/) (2021)
- [Recovering 3D Human Mesh from Monocular Images: A Survey](https://mlsanity.com/paper/2203.01923/) (2022)
- [SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesis](https://mlsanity.com/paper/2404.04104/) (2024)
- [H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction](https://mlsanity.com/paper/2107.12512/) (2021)
- [Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videos](https://mlsanity.com/paper/2207.11094/) (2022)
- [HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recovery](https://mlsanity.com/paper/2304.05690/) (2023)
- [FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction](https://mlsanity.com/paper/2111.01082/) (2021)
- [Instant Multi-View Head Capture through Learnable Registration](https://mlsanity.com/paper/2306.07437/) (2023)
- [Deep Learning-based Face Super-Resolution: A Survey](https://mlsanity.com/paper/2101.03749/) (2021)
- [SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction](https://mlsanity.com/paper/2106.03021/) (2021)
- [Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometry](https://mlsanity.com/paper/2110.09772/) (2021)
- [H3WB: Human3.6M 3D WholeBody Dataset and Benchmark](https://mlsanity.com/paper/2211.15692/) (2022)
- [HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details](https://mlsanity.com/paper/2303.11225/) (2023)
- [Self-Supervised 3D Face Reconstruction via Conditional Estimation](https://mlsanity.com/paper/2110.04800/) (2021)
- [AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypoints](https://mlsanity.com/paper/2205.10636/) (2022)
- [Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection](https://mlsanity.com/paper/2106.07852/) (2021)
- [Zero-Shot Text-to-Parameter Translation for Game Character Auto-Creation](https://mlsanity.com/paper/2303.01311/) (2023)
- [Image2Lego: Customized LEGO Set Generation from Images](https://mlsanity.com/paper/2108.08477/) (2021)
- [A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generation](https://mlsanity.com/paper/2406.04253/) (2024)
- [BareSkinNet: De-makeup and De-lighting via 3D Face Reconstruction](https://mlsanity.com/paper/2209.09029/) (2022)
- [SIDER: Single-Image Neural Optimization for Facial Geometric Detail Recovery](https://mlsanity.com/paper/2108.05465/) (2021)
- [Robust Egocentric Photo-realistic Facial Expression Transfer for Virtual Reality](https://mlsanity.com/paper/2104.04794/) (2021)
- [Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction](https://mlsanity.com/paper/2310.08784/) (2023)
- [Learning Free-Form Deformation for 3D Face Reconstruction from In-The-Wild Images](https://mlsanity.com/paper/2105.14857/) (2021)
- [Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions](https://mlsanity.com/paper/2209.04436/) (2022)
- [RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairs](https://mlsanity.com/paper/2302.05486/) (2023)
- [DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs](https://mlsanity.com/paper/2302.12253/) (2023)

## Abstract (arXiv)

3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. Current systems often assume the availability of multiple facial images (sometimes from the same subject) as input, and must address a number of methodological challenges such as establishing dense correspondences across large facial poses, expressions, and non-uniform illumination. In general these methods require complex and inefficient pipelines for model building and fitting. In this work, we propose to address many of these limitations by training a Convolutional Neural Network (CNN) on an appropriate dataset consisting of 2D images and 3D facial models or scans. Our CNN works with just a single 2D facial image, does not require accurate alignment nor establishes dense correspondence between images, works for arbitrary facial poses and expressions, and can be used to reconstruct the whole 3D facial geometry (including the non-visible parts of the face) bypassing the construction (during training) and fitting (during testing) of a 3D Morphable Model. We achieve this via a simple CNN architecture that performs direct regression of a volumetric representation of the 3D facial geometry from a single 2D image. We also demonstrate how the related task of facial landmark localization can be incorporated into the proposed framework and help improve reconstruction quality, especially for the cases of large poses and facial expressions. Testing code will be made available online, along with pre-trained models http://aaronsplace.co.uk/papers/jackson2017recon