# Lightweight Photometric Stereo for Facial Details Recovery

arXiv: https://arxiv.org/abs/2003.12307 · HTML: https://mlsanity.com/paper/2003.12307/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2020.

## Built on by (most cited first, 8 of 8)

- [AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis](https://mlsanity.com/paper/2103.11078/) (2021)
- [Deep Learning-based Face Super-Resolution: A Survey](https://mlsanity.com/paper/2101.03749/) (2021)
- [Prior-Guided Multi-View 3D Head Reconstruction](https://mlsanity.com/paper/2107.04277/) (2021)
- [High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptation](https://mlsanity.com/paper/2103.15876/) (2021)
- [BabyNet: Reconstructing 3D faces of babies from uncalibrated photographs](https://mlsanity.com/paper/2203.05908/) (2022)
- [Self-supervised Re-renderable Facial Albedo Reconstruction from Single Image](https://mlsanity.com/paper/2111.08282/) (2021)
- [3Deformer: A Common Framework for Image-Guided Mesh Deformation](https://mlsanity.com/paper/2307.09892/) (2023)
- [HeadRecon: High-Fidelity 3D Head Reconstruction from Monocular Video](https://mlsanity.com/paper/2312.08863/) (2023)

## Abstract (arXiv)

Recently, 3D face reconstruction from a single image has achieved great success with the help of deep learning and shape prior knowledge, but they often fail to produce accurate geometry details. On the other hand, photometric stereo methods can recover reliable geometry details, but require dense inputs and need to solve a complex optimization problem. In this paper, we present a lightweight strategy that only requires sparse inputs or even a single image to recover high-fidelity face shapes with images captured under near-field lights. To this end, we construct a dataset containing 84 different subjects with 29 expressions under 3 different lights. Data augmentation is applied to enrich the data in terms of diversity in identity, lighting, expression, etc. With this constructed dataset, we propose a novel neural network specially designed for photometric stereo based 3D face reconstruction. Extensive experiments and comparisons demonstrate that our method can generate high-quality reconstruction results with one to three facial images captured under near-field lights. Our full framework is available at https://github.com/Juyong/FacePSNet.