# Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression

arXiv: https://arxiv.org/abs/2208.07142 · HTML: https://mlsanity.com/paper/2208.07142/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 15 Aug 2022.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021)
- [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018)
- [Towards Metrical Reconstruction of Human Faces](https://mlsanity.com/paper/2204.06607/) (2022)
- [Nonlinear 3D Face Morphable Model](https://mlsanity.com/paper/1804.03786/) (2018)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network](https://mlsanity.com/paper/1612.04904/) (2016)
- [Unsupervised Training for 3D Morphable Model Regression](https://mlsanity.com/paper/1806.06098/) (2018)
- [Sample and Computation Redistribution for Efficient Face Detection](https://mlsanity.com/paper/2105.04714/) (2021)
- [Killing Two Birds with One Stone:Efficient and Robust Training of Face Recognition CNNs by Partial FC](https://mlsanity.com/paper/2203.15565/) (2022)
- [3D face reconstruction with dense landmarks](https://mlsanity.com/paper/2204.02776/) (2022)
- [Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression](https://mlsanity.com/paper/1703.07834/) (2017)
- [WebFace260M: A Benchmark for Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2204.10149/) (2022)
- [DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild](https://mlsanity.com/paper/1803.02188/) (2018)
- [Dense 3D Face Decoding over 2500FPS: Joint Texture & Shape Convolutional Mesh Decoders](https://mlsanity.com/paper/1904.03525/) (2019)
- [Joint Face Detection and Facial Motion Retargeting for Multiple Faces](https://mlsanity.com/paper/1902.10744/) (2019)

## Built on by (most cited first, 4 of 4)

- [ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignment](https://mlsanity.com/paper/2308.14448/) (2023)
- [Real-time 3D-aware Portrait Editing from a Single Image](https://mlsanity.com/paper/2402.14000/) (2024)
- [Bidirectional Regression for Monocular 6DoF Head Pose Estimation and Reference System Alignment](https://mlsanity.com/paper/2407.14136/) (2024)
- [Diffusion-Based Makeup Transfer with Facial Region-Aware Makeup Features](https://mlsanity.com/paper/2603.20012/) (2026)

## Abstract (arXiv)

Even though 3D face reconstruction has achieved impressive progress, most orthogonal projection-based face reconstruction methods can not achieve accurate and consistent reconstruction results when the face is very close to the camera due to the distortion under the perspective projection. In this paper, we propose to simultaneously reconstruct 3D face mesh in the world space and predict 2D face landmarks on the image plane to address the problem of perspective 3D face reconstruction. Based on the predicted 3D vertices and 2D landmarks, the 6DoF (6 Degrees of Freedom) face pose can be easily estimated by the PnP solver to represent perspective projection. Our approach achieves 1st place on the leader-board of the ECCV 2022 WCPA challenge and our model is visually robust under different identities, expressions and poses. The training code and models are released to facilitate future research.