# Facial Appearance Capture at Home with Patch-Level Reflectance Prior

arXiv: https://arxiv.org/abs/2506.03478 · HTML: https://mlsanity.com/paper/2506.03478/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 27 of its 60 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 27 of 27)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](https://mlsanity.com/paper/2302.08113/) (2023)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [ReconFusion: 3D Reconstruction with Diffusion Priors](https://mlsanity.com/paper/2312.02981/) (2023)
- [Learning Formation of Physically-Based Face Attributes](https://mlsanity.com/paper/2004.03458/) (2020)
- [AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"](https://mlsanity.com/paper/2003.13845/) (2020)
- [DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance](https://mlsanity.com/paper/2304.03117/) (2023)
- [Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusion](https://mlsanity.com/paper/2312.00852/) (2023)
- [IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination](https://mlsanity.com/paper/2404.11593/) (2024)
- [Mixture of Diffusers for scene composition and high resolution image generation](https://mlsanity.com/paper/2302.02412/) (2023)
- [A Morphable Face Albedo Model](https://mlsanity.com/paper/2004.02711/) (2020)
- [AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs](https://mlsanity.com/paper/2112.05957/) (2021)
- [High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies](https://mlsanity.com/paper/2010.05562/) (2020)
- [Photo-Realistic Facial Details Synthesis from Single Image](https://mlsanity.com/paper/1903.10873/) (2019)
- [FLARE: Fast Learning of Animatable and Relightable Mesh Avatars](https://mlsanity.com/paper/2310.17519/) (2023)
- [Relightify: Relightable 3D Faces from a Single Image via Diffusion Models](https://mlsanity.com/paper/2305.06077/) (2023)
- [High-Res Facial Appearance Capture from Polarized Smartphone Images](https://mlsanity.com/paper/2212.01160/) (2022)
- [MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shading](https://mlsanity.com/paper/2312.13091/) (2023)
- [High-Quality Facial Geometry and Appearance Capture at Home](https://mlsanity.com/paper/2312.03442/) (2023)
- [HACK: Learning a Parametric Head and Neck Model for High-fidelity Animation](https://mlsanity.com/paper/2305.04469/) (2023)

## Built on by (most cited first, 5 of 5)

- [VGGTFace: Topologically Consistent Facial Geometry Reconstruction in the Wild](https://mlsanity.com/paper/2511.20366/) (2025)
- [WildCap: Facial Albedo Capture in the Wild via Hybrid Inverse Rendering](https://mlsanity.com/paper/2512.11237/) (2025)
- [RelightAnyone: A Generalized Relightable 3D Gaussian Head Model](https://mlsanity.com/paper/2601.03357/) (2026)
- [Learning a Delighting Prior for Facial Appearance Capture in the Wild](https://mlsanity.com/paper/2605.05636/) (2026)
- [TOPOS: High-Fidelity and Efficient Industry-Grade 3D Head Generation](https://mlsanity.com/paper/2605.14594/) (2026)

## Abstract (arXiv)

Existing facial appearance capture methods can reconstruct plausible facial reflectance from smartphone-recorded videos. However, the reconstruction quality is still far behind the ones based on studio recordings. This paper fills the gap by developing a novel daily-used solution with a co-located smartphone and flashlight video capture setting in a dim room. To enhance the quality, our key observation is to solve facial reflectance maps within the data distribution of studio-scanned ones. Specifically, we first learn a diffusion prior over the Light Stage scans and then steer it to produce the reflectance map that best matches the captured images. We propose to train the diffusion prior at the patch level to improve generalization ability and training stability, as current Light Stage datasets are in ultra-high resolution but limited in data size. Tailored to this prior, we propose a patch-level posterior sampling technique to sample seamless full-resolution reflectance maps from this patch-level diffusion model. Experiments demonstrate our method closes the quality gap between low-cost and studio recordings by a large margin, opening the door for everyday users to clone themselves to the digital world. Our code will be released at https://github.com/yxuhan/DoRA.