# Recognizing Material Properties from Images

arXiv: https://arxiv.org/abs/1801.03127 · HTML: https://mlsanity.com/paper/1801.03127/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2018.

## Built on by (most cited first, 14 of 14)

- [Materialistic: Selecting Similar Materials in Images](https://mlsanity.com/paper/2305.13291/) (2023)
- [A Dense Material Segmentation Dataset for Indoor and Outdoor Scene Parsing](https://mlsanity.com/paper/2207.10614/) (2022)
- [SAMa: Material-aware 3D Selection and Segmentation](https://mlsanity.com/paper/2411.19322/) (2024)
- [One-shot recognition of any material anywhere using contrastive learning with physics-based rendering](https://mlsanity.com/paper/2212.00648/) (2022)
- [Hierarchical Material Recognition from Local Appearance](https://mlsanity.com/paper/2505.22911/) (2025)
- [Material Measurement Units for a Circular Economy: Foundations through a Review](https://mlsanity.com/paper/2103.01997/) (2021)
- [MatPredict: a dataset and benchmark for learning material properties of diverse indoor objects](https://mlsanity.com/paper/2505.13201/) (2025)
- [Material Fingerprinting: Identifying and Predicting Perceptual Attributes of Material Appearance](https://mlsanity.com/paper/2410.13615/) (2024)
- [DBAT: Dynamic Backward Attention Transformer for Material Segmentation with Cross-Resolution Patches](https://mlsanity.com/paper/2305.03919/) (2023)
- [MatSpectNet: Material Segmentation Network with Domain-Aware and Physically-Constrained Hyperspectral Reconstruction](https://mlsanity.com/paper/2307.11466/) (2023)
- [Visual Material Characteristics Learning for Circular Healthcare](https://mlsanity.com/paper/2309.04763/) (2023)
- [LLM-Guided Material Inference for 3D Point Clouds](https://mlsanity.com/paper/2512.03237/) (2025)
- [Monocular Reconstruction of Neural Tactile Fields](https://mlsanity.com/paper/2602.12508/) (2026)
- [BIMScript: Material-Aware Structured Scene Programs for BIM Ingestion](https://mlsanity.com/paper/2608.21447/) (2026)

## Abstract (arXiv)

Humans rely on properties of the materials that make up objects to guide our interactions with them. Grasping smooth materials, for example, requires care, and softness is an ideal property for fabric used in bedding. Even when these properties are not visual (e.g. softness is a physical property), we may still infer their presence visually. We refer to such material properties as visual material attributes. Recognizing these attributes in images can contribute valuable information for general scene understanding and material recognition. Unlike well-known object and scene attributes, visual material attributes are local properties with no fixed shape or spatial extent. We show that given a set of images annotated with known material attributes, we may accurately recognize the attributes from small local image patches. Obtaining such annotations in a consistent fashion at scale, however, is challenging. To address this, we introduce a method that allows us to probe the human visual perception of materials by asking simple yes/no questions comparing pairs of image patches. This provides sufficient weak supervision to build a set of attributes and associated classifiers that, while unnamed, serve the same function as the named attributes we use to describe materials. Doing so allows us to recognize visual material attributes without resorting to exhaustive manual annotation of a fixed set of named attributes. Furthermore, we show that this method may be integrated in the end-to-end learning of a material classification CNN to simultaneously recognize materials and discover their visual attributes. Our experimental results show that visual material attributes, whether named or automatically discovered, provide a useful intermediate representation for known material categories themselves as well as a basis for transfer learning when recognizing previously-unseen categories.