Monocular Facial Appearance Capture in the Wild
Not on the map. Why?
Abstract (arXiv)
We present a new method for reconstructing the appearance properties of human faces from a lightweight capture procedure in an unconstrained environment. Our method recovers the surface geometry, diffuse albedo, specular intensity and specular roughness from a monocular video containing a simple head rotation in-the-wild. Notably, we make no simplifying assumptions on the environment lighting, and we explicitly take visibility and occlusions into account. As a result, our method can produce facial appearance maps that approach the fidelity of studio-based multi-view captures, but with a far easier and cheaper procedure.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 40 of its 72 references that are in the map. First posted on arXiv 17 Dec 2024.
Builds on
40 of 72 references.
References · 40
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
- 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 496
- 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 477
- 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 449
- 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
- 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
- 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 335
- 2023GS-IR: 3D Gaussian Splatting for Inverse Renderingcited by 224
28 more
- 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
- 2021Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decompositioncited by 205
- 2017Morphable Face Models - An Open Frameworkcited by 138
- 2020MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decompositioncited by 127
- 2023Relightable Gaussian Codec Avatarscited by 127
- 2020AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"cited by 91
- 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89
- 2024IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illuminationcited by 67
- 2020A Morphable Face Albedo Modelcited by 54
- 2021Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracingcited by 48
- 2024GS^3: Efficient Relighting with Triple Gaussian Splattingcited by 44
- 2024DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shadingcited by 41
- 2023FLARE: Fast Learning of Animatable and Relightable Mesh Avatarscited by 39
- 2021Practical Face Reconstruction via Differentiable Ray Tracingcited by 35
- 2023Towards Practical Capture of High-Fidelity Relightable Avatarscited by 34
- 2022SunStage: Portrait Reconstruction and Relighting using the Sun as a Light Stagecited by 30
- 2024Photorealistic Object Insertion with Diffusion-Guided Inverse Renderingcited by 27
- 2022High-Res Facial Appearance Capture from Polarized Smartphone Imagescited by 21
- 2023Learning a 3D Morphable Face Reflectance Model from Low-cost Datacited by 20
- 2023MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shadingcited by 18
- 2024SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wildcited by 18
- 2022S2F2: Self-Supervised High Fidelity Face Reconstruction from Monocular Imagecited by 17
- 2023NeuFace: Realistic 3D Neural Face Rendering from Multi-view Imagescited by 12
- 2023Diffusion Reflectance Map: Single-Image Stochastic Inverse Rendering of Illumination and Reflectancecited by 11
- 2023High-Quality Facial Geometry and Appearance Capture at Homecited by 9
- 2023Artist-Friendly Relightable and Animatable Neural Headscited by 6
- 2024SPARK: Self-supervised Personalized Real-time Monocular Face Capturecited by 6
- 2023Differentiable Display Photometric Stereocited by 4
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.