# The Visual Language of Fabrics

arXiv: https://arxiv.org/abs/2307.13681 · HTML: https://mlsanity.com/paper/2307.13681/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 69 references that are in the map.
- First posted on arXiv 25 Jul 2023.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [TANGO: Text-driven Photorealistic and Robust 3D Stylization via Lighting Decomposition](https://mlsanity.com/paper/2210.11277/) (2022)
- [Learning Intrinsic Image Decomposition from Watching the World](https://mlsanity.com/paper/1804.00582/) (2018)
- [MaterialGAN: Reflectance Capture using a Generative SVBRDF Model](https://mlsanity.com/paper/2010.00114/) (2020)
- [TileGen: Tileable, Controllable Material Generation and Capture](https://mlsanity.com/paper/2206.05649/) (2022)
- [A Similarity Measure for Material Appearance](https://mlsanity.com/paper/1905.01562/) (2019)
- [An intuitive control space for material appearance](https://mlsanity.com/paper/1806.04950/) (2018)
- [Describing Textures using Natural Language](https://mlsanity.com/paper/2008.01180/) (2020)
- [The joint role of geometry and illumination on material recognition](https://mlsanity.com/paper/2101.02496/) (2021)

## Built on by (most cited first, 7 of 7)

- [MatSynth: A Modern PBR Materials Dataset](https://mlsanity.com/paper/2401.06056/) (2024)
- [TexSliders: Diffusion-Based Texture Editing in CLIP Space](https://mlsanity.com/paper/2405.00672/) (2024)
- [MatAtlas: Text-driven Consistent Geometry Texturing and Material Assignment](https://mlsanity.com/paper/2404.02899/) (2024)
- [MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models](https://mlsanity.com/paper/2501.15981/) (2025)
- [Material Fingerprinting: Identifying and Predicting Perceptual Attributes of Material Appearance](https://mlsanity.com/paper/2410.13615/) (2024)
- [MAOAM: Unified Object and Material Selection with Vision-Language Models](https://mlsanity.com/paper/2606.04880/) (2026)
- [MatLoom: Layered Text-to-Material Generation in a Compact Program Space](https://mlsanity.com/paper/2609.40322/) (2026)

## Abstract (arXiv)

We introduce text2fabric, a novel dataset that links free-text descriptions to various fabric materials. The dataset comprises 15,000 natural language descriptions associated to 3,000 corresponding images of fabric materials. Traditionally, material descriptions come in the form of tags/keywords, which limits their expressivity, induces pre-existing knowledge of the appropriate vocabulary, and ultimately leads to a chopped description system. Therefore, we study the use of free-text as a more appropriate way to describe material appearance, taking the use case of fabrics as a common item that non-experts may often deal with. Based on the analysis of the dataset, we identify a compact lexicon, set of attributes and key structure that emerge from the descriptions. This allows us to accurately understand how people describe fabrics and draw directions for generalization to other types of materials. We also show that our dataset enables specializing large vision-language models such as CLIP, creating a meaningful latent space for fabric appearance, and significantly improving applications such as fine-grained material retrieval and automatic captioning.