# Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function

arXiv: https://arxiv.org/abs/2201.01016 · HTML: https://mlsanity.com/paper/2201.01016/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 53 references that are in the map.
- First posted on arXiv 4 Jan 2022.

## Builds on (most cited first, 25 of 25)

- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction](https://mlsanity.com/paper/2003.13989/) (2020)
- [Learning a Multi-View Stereo Machine](https://mlsanity.com/paper/1708.05375/) (2017)
- [Tex2Shape: Detailed Full Human Body Geometry From a Single Image](https://mlsanity.com/paper/1904.08645/) (2019)
- [Point-Based Multi-View Stereo Network](https://mlsanity.com/paper/1908.04422/) (2019)
- [SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsis](https://mlsanity.com/paper/1708.01749/) (2017)
- [DPSNet: End-to-end Deep Plane Sweep Stereo](https://mlsanity.com/paper/1905.00538/) (2019)
- [Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking](https://mlsanity.com/paper/2007.10872/) (2020)
- [Detailed Human Shape Estimation from a Single Image by Hierarchical Mesh Deformation](https://mlsanity.com/paper/1904.10506/) (2019)
- [Pyramid Multi-view Stereo Net with Self-adaptive View Aggregation](https://mlsanity.com/paper/1912.03001/) (2019)
- [FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction](https://mlsanity.com/paper/2111.01082/) (2021)
- [Riggable 3D Face Reconstruction via In-Network Optimization](https://mlsanity.com/paper/2104.03493/) (2021)
- [Learned Multi-Patch Similarity](https://mlsanity.com/paper/1703.08836/) (2017)
- [Detailed Avatar Recovery from Single Image](https://mlsanity.com/paper/2108.02931/) (2021)

## Built on by (most cited first, 14 of 14)

- [A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Images](https://mlsanity.com/paper/2302.14434/) (2023)
- [AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation](https://mlsanity.com/paper/2306.09864/) (2023)
- [HHAvatar: Gaussian Head Avatar with Dynamic Hairs](https://mlsanity.com/paper/2312.03029/) (2023)
- [High-Fidelity 3D Face Generation from Natural Language Descriptions](https://mlsanity.com/paper/2305.03302/) (2023)
- [Head360: Learning a Parametric 3D Full-Head for Free-View Synthesis in 360deg](https://mlsanity.com/paper/2408.00296/) (2024)
- [Topo4D: Topology-Preserving Gaussian Splatting for High-Fidelity 4D Head Capture](https://mlsanity.com/paper/2406.00440/) (2024)
- [RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairs](https://mlsanity.com/paper/2302.05486/) (2023)
- [Towards Native Generative Model for 3D Head Avatar](https://mlsanity.com/paper/2410.01226/) (2024)
- [SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling](https://mlsanity.com/paper/2307.00804/) (2023)
- [LLM-to-Phy3D: Physically Conform Online 3D Object Generation with LLMs](https://mlsanity.com/paper/2506.11148/) (2025)
- [VGGTFace: Topologically Consistent Facial Geometry Reconstruction in the Wild](https://mlsanity.com/paper/2511.20366/) (2025)
- [From Blurry to Believable: Enhancing Low-quality Talking Heads with 3D Generative Priors](https://mlsanity.com/paper/2602.06122/) (2026)
- [Registration-Free Learnable Multi-View Capture of Faces in Dense Semantic Correspondence](https://mlsanity.com/paper/2605.01450/) (2026)
- [From 2D Images to 3D Model:Weakly Supervised Multi-View Face Reconstruction with Deep Fusion](https://mlsanity.com/paper/2204.03842/) (2022)

## Abstract (arXiv)

Recovering detailed facial geometry from a set of calibrated multi-view images is valuable for its wide range of applications. Traditional multi-view stereo (MVS) methods adopt an optimization-based scheme to regularize the matching cost. Recently, learning-based methods integrate all these into an end-to-end neural network and show superiority of efficiency. In this paper, we propose a novel architecture to recover extremely detailed 3D faces within dozens of seconds. Unlike previous learning-based methods that regularize the cost volume via 3D CNN, we propose to learn an implicit function for regressing the matching cost. By fitting a 3D morphable model from multi-view images, the features of multiple images are extracted and aggregated in the mesh-attached UV space, which makes the implicit function more effective in recovering detailed facial shape. Our method outperforms SOTA learning-based MVS in accuracy by a large margin on the FaceScape dataset. The code and data are released in https://github.com/zhuhao-nju/mvfr.