# Material Recognition from Local Appearance in Global Context

arXiv: https://arxiv.org/abs/1611.09394 · HTML: https://mlsanity.com/paper/1611.09394/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2016.

## Built on by (most cited first, 5 of 5)

- [Materialistic: Selecting Similar Materials in Images](https://mlsanity.com/paper/2305.13291/) (2023)
- [Hierarchical Material Recognition from Local Appearance](https://mlsanity.com/paper/2505.22911/) (2025)
- [Object-Oriented Material Classification and 3D Clustering for Improved Semantic Perception and Mapping in Mobile Robots](https://mlsanity.com/paper/2407.06077/) (2024)
- [DBAT: Dynamic Backward Attention Transformer for Material Segmentation with Cross-Resolution Patches](https://mlsanity.com/paper/2305.03919/) (2023)
- [MatSpectNet: Material Segmentation Network with Domain-Aware and Physically-Constrained Hyperspectral Reconstruction](https://mlsanity.com/paper/2307.11466/) (2023)

## Abstract (arXiv)

Recognition of materials has proven to be a challenging problem due to the wide variation in appearance within and between categories. Global image context, such as where the material is or what object it makes up, can be crucial to recognizing the material. Existing methods, however, operate on an implicit fusion of materials and context by using large receptive fields as input (i.e., large image patches). Many recent material recognition methods treat materials as yet another set of labels like objects. Materials are, however, fundamentally different from objects as they have no inherent shape or defined spatial extent. Approaches that ignore this can only take advantage of limited implicit context as it appears during training. We instead show that recognizing materials purely from their local appearance and integrating separately recognized global contextual cues including objects and places leads to superior dense, per-pixel, material recognition. We achieve this by training a fully-convolutional material recognition network end-to-end with only material category supervision. We integrate object and place estimates to this network from independent CNNs. This approach avoids the necessity of preparing an impractically-large amount of training data to cover the product space of materials, objects, and scenes, while fully leveraging contextual cues for dense material recognition. Furthermore, we perform a detailed analysis of the effects of context granularity, spatial resolution, and the network level at which we introduce context. On a recently introduced comprehensive and diverse material database citeSchwartz2016, we confirm that our method achieves state-of-the-art accuracy with significantly less training data compared to past methods.