# HumanMaterial: Human Material Estimation from a Single Image via Progressive Training

arXiv: https://arxiv.org/abs/2507.18385 · HTML: https://mlsanity.com/paper/2507.18385/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Builds on (most cited first, 27 of 27)

- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Modeling Indirect Illumination for Inverse Rendering](https://mlsanity.com/paper/2204.06837/) (2022)
- [TensoIR: Tensorial Inverse Rendering](https://mlsanity.com/paper/2304.12461/) (2023)
- [NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images](https://mlsanity.com/paper/2305.17398/) (2023)
- [Single Image Portrait Relighting](https://mlsanity.com/paper/1905.00824/) (2019)
- [Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothing](https://mlsanity.com/paper/2204.08906/) (2022)
- [Single-Image SVBRDF Capture with a Rendering-Aware Deep Network](https://mlsanity.com/paper/1810.09718/) (2018)
- [NeILF: Neural Incident Light Field for Physically-based Material Estimation](https://mlsanity.com/paper/2203.07182/) (2022)
- [AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"](https://mlsanity.com/paper/2003.13845/) (2020)
- [SwitchLight: Co-design of Physics-driven Architecture and Pre-training Framework for Human Portrait Relighting](https://mlsanity.com/paper/2402.18848/) (2024)
- [Unsupervised Learning for Intrinsic Image Decomposition from a Single Image](https://mlsanity.com/paper/1911.09930/) (2019)
- [Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networks](https://mlsanity.com/paper/1809.00886/) (2018)
- [AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs](https://mlsanity.com/paper/2112.05957/) (2021)
- [Relighting4D: Neural Relightable Human from Videos](https://mlsanity.com/paper/2207.07104/) (2022)
- [Relightify: Relightable 3D Faces from a Single Image via Diffusion Models](https://mlsanity.com/paper/2305.06077/) (2023)
- [NeuLF: Efficient Novel View Synthesis with Neural 4D Light Field](https://mlsanity.com/paper/2105.07112/) (2021)
- [Single-image Full-body Human Relighting](https://mlsanity.com/paper/2107.07259/) (2021)
- [Relightable and Animatable Neural Avatar from Sparse-View Video](https://mlsanity.com/paper/2308.07903/) (2023)
- [High-Res Facial Appearance Capture from Polarized Smartphone Images](https://mlsanity.com/paper/2212.01160/) (2022)
- [Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects](https://mlsanity.com/paper/2305.17929/) (2023)
- [DPCS: Path Tracing-Based Differentiable Projector-Camera Systems](https://mlsanity.com/paper/2503.12174/) (2025)
- [Efficient Reflectance Capture with a Deep Gated Mixture-of-Experts](https://mlsanity.com/paper/2203.15258/) (2022)
- [FROST-BRDF: A Fast and Robust Optimal Sampling Technique for BRDF Acquisition](https://mlsanity.com/paper/2401.07283/) (2024)
- [All-frequency Full-body Human Image Relighting](https://mlsanity.com/paper/2411.00356/) (2024)

## Built on by (most cited first, 1 of 1)

- [HAFMat: Hybrid Priors Guided Adaptive Fusion for Single-Image Human Material Estimation](https://mlsanity.com/paper/2606.16323/) (2026)

## Abstract (arXiv)

Full-body Human inverse rendering based on physically-based rendering aims to acquire high-quality materials, which helps achieve photo-realistic rendering under arbitrary illuminations. This task requires estimating multiple material maps and usually relies on the constraint of rendering result. The absence of constraints on the material maps makes inverse rendering an ill-posed task. Previous works alleviated this problem by building material dataset for training, but their simplified material data and rendering equation lead to rendering results with limited realism, especially that of skin. To further alleviate this problem, we construct a higher-quality dataset (OpenHumanBRDF) based on scanned real data and statistical material data. In addition to the normal, diffuse albedo, roughness, specular albedo, we produce displacement and subsurface scattering to enhance the realism of rendering results, especially for the skin. With the increase in prediction tasks for more materials, using an end-to-end model as in the previous work struggles to balance the importance among various material maps, and leads to model underfitting. Therefore, we design a model (HumanMaterial) with progressive training strategy to make full use of the supervision information of the material maps and improve the performance of material estimation. HumanMaterial first obtain the initial material results via three prior models, and then refine the results by a finetuning model. Prior models estimate different material maps, and each map has different significance for rendering results. Thus, we design a Controlled PBR Rendering (CPR) loss, which enhances the importance of the materials to be optimized during the training of prior models. Extensive experiments on OpenHumanBRDF dataset and real data demonstrate that our method achieves state-of-the-art performance.