# Generative Modelling of BRDF Textures from Flash Images

arXiv: https://arxiv.org/abs/2102.11861 · HTML: https://mlsanity.com/paper/2102.11861/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 68 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [SinGAN: Learning a Generative Model from a Single Natural Image](https://mlsanity.com/paper/1905.01164/) (2019)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images](https://mlsanity.com/paper/1603.03417/) (2016)
- [Single-Image SVBRDF Capture with a Rendering-Aware Deep Network](https://mlsanity.com/paper/1810.09718/) (2018)
- [Non-Stationary Texture Synthesis by Adversarial Expansion](https://mlsanity.com/paper/1805.04487/) (2018)
- [Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image](https://mlsanity.com/paper/1804.05790/) (2018)
- [Two-shot Spatially-varying BRDF and Shape Estimation](https://mlsanity.com/paper/2004.00403/) (2020)
- [Learning Texture Manifolds with the Periodic Spatial GAN](https://mlsanity.com/paper/1705.06566/) (2017)
- [Learning a Neural 3D Texture Space from 2D Exemplars](https://mlsanity.com/paper/1912.04158/) (2019)
- [Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networks](https://mlsanity.com/paper/1809.00886/) (2018)
- [Flexible SVBRDF Capture with a Multi-Image Deep Network](https://mlsanity.com/paper/1906.11557/) (2019)
- [MaterialGAN: Reflectance Capture using a Generative SVBRDF Model](https://mlsanity.com/paper/2010.00114/) (2020)
- [Guided Fine-Tuning for Large-Scale Material Transfer](https://mlsanity.com/paper/2007.03059/) (2020)
- [Material Editing Using a Physically Based Rendering Network](https://mlsanity.com/paper/1708.00106/) (2017)
- [Synthesizing 3D Shapes from Silhouette Image Collections using Multi-projection Generative Adversarial Networks](https://mlsanity.com/paper/1906.03841/) (2019)
- [Deep Reflectance Maps](https://mlsanity.com/paper/1511.04384/) (2015)
- [A Bayesian Inference Framework for Procedural Material Parameter Estimation](https://mlsanity.com/paper/1912.01067/) (2019)
- [An intuitive control space for material appearance](https://mlsanity.com/paper/1806.04950/) (2018)
- [Texture Synthesis Through Convolutional Neural Networks and Spectrum Constraints](https://mlsanity.com/paper/1605.01141/) (2016)
- [Gaussian Material Synthesis](https://mlsanity.com/paper/1804.08369/) (2018)

## Built on by (most cited first, 30 of 35)

- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections](https://mlsanity.com/paper/2205.15768/) (2022)
- [ControlMat: A Controlled Generative Approach to Material Capture](https://mlsanity.com/paper/2309.01700/) (2023)
- [MatSynth: A Modern PBR Materials Dataset](https://mlsanity.com/paper/2401.06056/) (2024)
- [MatFusion: A Generative Diffusion Model for SVBRDF Capture](https://mlsanity.com/paper/2406.06539/) (2024)
- [TileGen: Tileable, Controllable Material Generation and Capture](https://mlsanity.com/paper/2206.05649/) (2022)
- [Materialistic: Selecting Similar Materials in Images](https://mlsanity.com/paper/2305.13291/) (2023)
- [PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes](https://mlsanity.com/paper/2207.00757/) (2022)
- [BlenderAlchemy: Editing 3D Graphics with Vision-Language Models](https://mlsanity.com/paper/2404.17672/) (2024)
- [SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps](https://mlsanity.com/paper/2201.05120/) (2022)
- [PhotoMat: A Material Generator Learned from Single Flash Photos](https://mlsanity.com/paper/2305.12296/) (2023)
- [Objects With Lighting: A Real-World Dataset for Evaluating Reconstruction and Rendering for Object Relighting](https://mlsanity.com/paper/2401.09126/) (2024)
- [SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild](https://mlsanity.com/paper/2401.10171/) (2024)
- [UMat: Uncertainty-Aware Single Image High Resolution Material Capture](https://mlsanity.com/paper/2305.16312/) (2023)
- [Learning to Learn and Sample BRDFs](https://mlsanity.com/paper/2210.03510/) (2022)
- [VLMaterial: Procedural Material Generation with Large Vision-Language Models](https://mlsanity.com/paper/2501.18623/) (2025)
- [SViM3D: Stable Video Material Diffusion for Single Image 3D Generation](https://mlsanity.com/paper/2510.08271/) (2025)
- [Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Images](https://mlsanity.com/paper/2509.09952/) (2025)
- [SVBRDF Recovery From a Single Image With Highlights using a Pretrained Generative Adversarial Network](https://mlsanity.com/paper/2111.00943/) (2021)
- [Uncertainty for SVBRDF Acquisition using Frequency Analysis](https://mlsanity.com/paper/2406.17774/) (2024)
- [Efficient Reflectance Capture with a Deep Gated Mixture-of-Experts](https://mlsanity.com/paper/2203.15258/) (2022)
- [Retargeting Visual Data with Deformation Fields](https://mlsanity.com/paper/2311.13297/) (2023)
- [FROST-BRDF: A Fast and Robust Optimal Sampling Technique for BRDF Acquisition](https://mlsanity.com/paper/2401.07283/) (2024)
- [Real-time Virtual-Try-On from a Single Example Image through Deep Inverse Graphics and Learned Differentiable Renderers](https://mlsanity.com/paper/2205.06305/) (2022)
- [Tree-Structured Shading Decomposition](https://mlsanity.com/paper/2309.07122/) (2023)
- [DreamPBR: Text-driven Generation of High-resolution SVBRDF with Multi-modal Guidance](https://mlsanity.com/paper/2404.14676/) (2024)
- [Content-aware Tile Generation using Exterior Boundary Inpainting](https://mlsanity.com/paper/2409.14184/) (2024)
- [On Neural BRDFs: A Thorough Comparison of State-of-the-Art Approaches](https://mlsanity.com/paper/2502.15480/) (2025)
- [FabricDiffusion: High-Fidelity Texture Transfer for 3D Garments Generation from In-The-Wild Clothing Images](https://mlsanity.com/paper/2410.01801/) (2024)
- [Neural Differential Appearance Equations](https://mlsanity.com/paper/2410.07128/) (2024)

## Abstract (arXiv)

We learn a latent space for easy capture, consistent interpolation, and efficient reproduction of visual material appearance. When users provide a photo of a stationary natural material captured under flashlight illumination, first it is converted into a latent material code. Then, in the second step, conditioned on the material code, our method produces an infinite and diverse spatial field of BRDF model parameters (diffuse albedo, normals, roughness, specular albedo) that subsequently allows rendering in complex scenes and illuminations, matching the appearance of the input photograph. Technically, we jointly embed all flash images into a latent space using a convolutional encoder, and -- conditioned on these latent codes -- convert random spatial fields into fields of BRDF parameters using a convolutional neural network (CNN). We condition these BRDF parameters to match the visual characteristics (statistics and spectra of visual features) of the input under matching light. A user study compares our approach favorably to previous work, even those with access to BRDF supervision.