# MaterialGAN: Reflectance Capture using a Generative SVBRDF Model

arXiv: https://arxiv.org/abs/2010.00114 · HTML: https://mlsanity.com/paper/2010.00114/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2020.

## Built on by (most cited first, 30 of 33)

- [NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images](https://mlsanity.com/paper/2305.17398/) (2023)
- [Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Rendering](https://mlsanity.com/paper/2103.15208/) (2021)
- [ControlMat: A Controlled Generative Approach to Material Capture](https://mlsanity.com/paper/2309.01700/) (2023)
- [DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models](https://mlsanity.com/paper/2405.17176/) (2024)
- [A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyond](https://mlsanity.com/paper/2112.03842/) (2021)
- [Generative Modelling of BRDF Textures from Flash Images](https://mlsanity.com/paper/2102.11861/) (2021)
- [MatFusion: A Generative Diffusion Model for SVBRDF Capture](https://mlsanity.com/paper/2406.06539/) (2024)
- [TileGen: Tileable, Controllable Material Generation and Capture](https://mlsanity.com/paper/2206.05649/) (2022)
- [Materialistic: Selecting Similar Materials in Images](https://mlsanity.com/paper/2305.13291/) (2023)
- [SurfaceNet: Adversarial SVBRDF Estimation from a Single Image](https://mlsanity.com/paper/2107.11298/) (2021)
- [PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes](https://mlsanity.com/paper/2207.00757/) (2022)
- [SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps](https://mlsanity.com/paper/2201.05120/) (2022)
- [PhotoMat: A Material Generator Learned from Single Flash Photos](https://mlsanity.com/paper/2305.12296/) (2023)
- [Uni-Renderer: Unifying Rendering and Inverse Rendering Via Dual Stream Diffusion](https://mlsanity.com/paper/2412.15050/) (2024)
- [UMat: Uncertainty-Aware Single Image High Resolution Material Capture](https://mlsanity.com/paper/2305.16312/) (2023)
- [Boosting 3D Object Generation through PBR Materials](https://mlsanity.com/paper/2411.16080/) (2024)
- [Learning to Learn and Sample BRDFs](https://mlsanity.com/paper/2210.03510/) (2022)
- [3inGAN: Learning a 3D Generative Model from Images of a Self-similar Scene](https://mlsanity.com/paper/2211.14902/) (2022)
- [The Visual Language of Fabrics](https://mlsanity.com/paper/2307.13681/) (2023)
- [Neural Photometry-guided Visual Attribute Transfer](https://mlsanity.com/paper/2112.02520/) (2021)
- [Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Images](https://mlsanity.com/paper/2509.09952/) (2025)
- [SVBRDF Recovery From a Single Image With Highlights using a Pretrained Generative Adversarial Network](https://mlsanity.com/paper/2111.00943/) (2021)
- [ePBR: Extended PBR Materials in Image Synthesis](https://mlsanity.com/paper/2504.17062/) (2025)
- [MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models](https://mlsanity.com/paper/2501.15981/) (2025)
- [ROSA: Reconstructing Object Shape and Appearance Textures by Adaptive Detail Transfer](https://mlsanity.com/paper/2501.18595/) (2025)
- [On Neural BRDFs: A Thorough Comparison of State-of-the-Art Approaches](https://mlsanity.com/paper/2502.15480/) (2025)
- [VideoNeuMat: Neural Material Extraction from Generative Video Models](https://mlsanity.com/paper/2602.07272/) (2026)
- [Neural Differential Appearance Equations](https://mlsanity.com/paper/2410.07128/) (2024)
- [Example-Based Feature Painting on Textures](https://mlsanity.com/paper/2511.01513/) (2025)
- [Deep scene-scale material estimation from multi-view indoor captures](https://mlsanity.com/paper/2211.08047/) (2022)

## Abstract (arXiv)

We address the problem of reconstructing spatially-varying BRDFs from a small set of image measurements. This is a fundamentally under-constrained problem, and previous work has relied on using various regularization priors or on capturing many images to produce plausible results. In this work, we present MaterialGAN, a deep generative convolutional network based on StyleGAN2, trained to synthesize realistic SVBRDF parameter maps. We show that MaterialGAN can be used as a powerful material prior in an inverse rendering framework: we optimize in its latent representation to generate material maps that match the appearance of the captured images when rendered. We demonstrate this framework on the task of reconstructing SVBRDFs from images captured under flash illumination using a hand-held mobile phone. Our method succeeds in producing plausible material maps that accurately reproduce the target images, and outperforms previous state-of-the-art material capture methods in evaluations on both synthetic and real data. Furthermore, our GAN-based latent space allows for high-level semantic material editing operations such as generating material variations and material morphing.