# Generating Parametric BRDFs from Natural Language Descriptions

arXiv: https://arxiv.org/abs/2306.15679 · HTML: https://mlsanity.com/paper/2306.15679/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 40 references that are in the map.
- First posted on arXiv 19 Jun 2023.

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Point-E: A System for Generating 3D Point Clouds from Complex Prompts](https://mlsanity.com/paper/2212.08751/) (2022)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [Text2Mesh: Text-Driven Neural Stylization for Meshes](https://mlsanity.com/paper/2112.03221/) (2021)
- [CLIP-Mesh: Generating textured meshes from text using pretrained image-text models](https://mlsanity.com/paper/2203.13333/) (2022)
- [RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation](https://mlsanity.com/paper/2211.09869/) (2022)
- [Compositional 3D Scene Generation using Locally Conditioned Diffusion](https://mlsanity.com/paper/2303.12218/) (2023)
- [TANGO: Text-driven Photorealistic and Robust 3D Stylization via Lighting Decomposition](https://mlsanity.com/paper/2210.11277/) (2022)
- [CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layout](https://mlsanity.com/paper/2303.13843/) (2023)
- [Neural BRDF Representation and Importance Sampling](https://mlsanity.com/paper/2102.05963/) (2021)
- [Generating Procedural Materials from Text or Image Prompts](https://mlsanity.com/paper/2304.13172/) (2023)
- [PhotoMat: A Material Generator Learned from Single Flash Photos](https://mlsanity.com/paper/2305.12296/) (2023)
- [Invertible Neural BRDF for Object Inverse Rendering](https://mlsanity.com/paper/2008.04030/) (2020)
- [Joint Representation Learning for Text and 3D Point Cloud](https://mlsanity.com/paper/2301.07584/) (2023)
- [Single Image BRDF Parameter Estimation with a Conditional Adversarial Network](https://mlsanity.com/paper/1910.05148/) (2019)
- [Describing Textures using Natural Language](https://mlsanity.com/paper/2008.01180/) (2020)
- [Interpretable Disentangled Parametrization of Measured BRDF with β-VAE](https://mlsanity.com/paper/2208.03914/) (2022)

## Built on by (most cited first, 3 of 3)

- [Towards Generalized and Training-Free Text-Guided Semantic Manipulation](https://mlsanity.com/paper/2504.17269/) (2025)
- [HiMat: DiT-based Ultra-High Resolution SVBRDF Generation](https://mlsanity.com/paper/2508.07011/) (2025)
- [Text-based Tactile Graphics Generation for the Visually Impaired](https://mlsanity.com/paper/2607.22674/) (2026)

## Abstract (arXiv)

Artistic authoring of 3D environments is a laborious enterprise that also requires skilled content creators. There have been impressive improvements in using machine learning to address different aspects of generating 3D content, such as generating meshes, arranging geometry, synthesizing textures, etc. In this paper we develop a model to generate Bidirectional Reflectance Distribution Functions (BRDFs) from descriptive textual prompts. BRDFs are four dimensional probability distributions that characterize the interaction of light with surface materials. They are either represented parametrically, or by tabulating the probability density associated with every pair of incident and outgoing angles. The former lends itself to artistic editing while the latter is used when measuring the appearance of real materials. Numerous works have focused on hypothesizing BRDF models from images of materials. We learn a mapping from textual descriptions of materials to parametric BRDFs. Our model is first trained using a semi-supervised approach before being tuned via an unsupervised scheme. Although our model is general, in this paper we specifically generate parameters for MDL materials, conditioned on natural language descriptions, within NVIDIA's Omniverse platform. This enables use cases such as real-time text prompts to change materials of objects in 3D environments such as "dull plastic" or "shiny iron". Since the output of our model is a parametric BRDF, rather than an image of the material, it may be used to render materials using any shape under arbitrarily specified viewing and lighting conditions.