# An Inverse Procedural Modeling Pipeline for SVBRDF Maps

arXiv: https://arxiv.org/abs/2109.06395 · HTML: https://mlsanity.com/paper/2109.06395/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2021.

## Built on by (most cited first, 24 of 24)

- [ControlMat: A Controlled Generative Approach to Material Capture](https://mlsanity.com/paper/2309.01700/) (2023)
- [Can Large Language Models Understand Symbolic Graphics Programs?](https://mlsanity.com/paper/2408.08313/) (2024)
- [MatFusion: A Generative Diffusion Model for SVBRDF Capture](https://mlsanity.com/paper/2406.06539/) (2024)
- [TileGen: Tileable, Controllable Material Generation and Capture](https://mlsanity.com/paper/2206.05649/) (2022)
- [MaPa: Text-driven Photorealistic Material Painting for 3D Shapes](https://mlsanity.com/paper/2404.17569/) (2024)
- [Materialistic: Selecting Similar Materials in Images](https://mlsanity.com/paper/2305.13291/) (2023)
- [ShapeCoder: Discovering Abstractions for Visual Programs from Unstructured Primitives](https://mlsanity.com/paper/2305.05661/) (2023)
- [PSDR-Room: Single Photo to Scene using Differentiable Rendering](https://mlsanity.com/paper/2307.03244/) (2023)
- [TexSliders: Diffusion-Based Texture Editing in CLIP Space](https://mlsanity.com/paper/2405.00672/) (2024)
- [PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes](https://mlsanity.com/paper/2207.00757/) (2022)
- [BlenderAlchemy: Editing 3D Graphics with Vision-Language Models](https://mlsanity.com/paper/2404.17672/) (2024)
- [Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis](https://mlsanity.com/paper/2412.08603/) (2024)
- [UMat: Uncertainty-Aware Single Image High Resolution Material Capture](https://mlsanity.com/paper/2305.16312/) (2023)
- [SAMa: Material-aware 3D Selection and Segmentation](https://mlsanity.com/paper/2411.19322/) (2024)
- [VLMaterial: Procedural Material Generation with Large Vision-Language Models](https://mlsanity.com/paper/2501.18623/) (2025)
- [Zero Grads: Learning Local Surrogate Losses for Non-Differentiable Graphics](https://mlsanity.com/paper/2308.05739/) (2023)
- [Tree-Structured Shading Decomposition](https://mlsanity.com/paper/2309.07122/) (2023)
- [DreamPBR: Text-driven Generation of High-resolution SVBRDF with Multi-modal Guidance](https://mlsanity.com/paper/2404.14676/) (2024)
- [SYNOSIS: Image synthesis pipeline for machine vision in metal surface inspection](https://mlsanity.com/paper/2410.14844/) (2024)
- [Single-view 3D reconstruction via inverse procedural modeling](https://mlsanity.com/paper/2310.13373/) (2023)
- [Single-image Reflectance and Transmittance Estimation from Any Flatbed Scanner](https://mlsanity.com/paper/2502.14462/) (2025)
- [RealMat: Realistic Materials with Diffusion and Reinforcement Learning](https://mlsanity.com/paper/2509.01134/) (2025)
- [Scene Co-pilot: Procedural Text to Video Generation with Human in the Loop](https://mlsanity.com/paper/2411.18644/) (2024)
- [DealMaTe: Multi-Dimensional Material Transfer via Diffusion Transformer](https://mlsanity.com/paper/2605.15681/) (2026)

## Abstract (arXiv)

Procedural modeling is now the de facto standard of material modeling in industry. Procedural models can be edited and are easily extended, unlike pixel-based representations of captured materials. In this paper, we present a semi-automatic pipeline for general material proceduralization. Given Spatially-Varying Bidirectional Reflectance Distribution Functions (SVBRDFs) represented as sets of pixel maps, our pipeline decomposes them into a tree of sub-materials whose spatial distributions are encoded by their associated mask maps. This semi-automatic decomposition of material maps progresses hierarchically, driven by our new spectrum-aware material matting and instance-based decomposition methods. Each decomposed sub-material is proceduralized by a novel multi-layer noise model to capture local variations at different scales. Spatial distributions of these sub-materials are modeled either by a by-example inverse synthesis method recovering Point Process Texture Basis Functions (PPTBF) or via random sampling. To reconstruct procedural material maps, we propose a differentiable rendering-based optimization that recomposes all generated procedures together to maximize the similarity between our procedural models and the input material pixel maps. We evaluate our pipeline on a variety of synthetic and real materials. We demonstrate our method's capacity to process a wide range of material types, eliminating the need for artist designed material graphs required in previous work. As fully procedural models, our results expand to arbitrary resolution and enable high level user control of appearance.