# Hierarchical Material Recognition from Local Appearance

arXiv: https://arxiv.org/abs/2505.22911 · HTML: https://mlsanity.com/paper/2505.22911/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 100 references that are in the map.
- First posted on arXiv 28 May 2025.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [EVA-02: A Visual Representation for Neon Genesis](https://mlsanity.com/paper/2303.11331/) (2023)
- [Few-Shot Learning with Graph Neural Networks](https://mlsanity.com/paper/1711.04043/) (2017)
- [Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs](https://mlsanity.com/paper/1803.08035/) (2018)
- [Rethinking Knowledge Graph Propagation for Zero-Shot Learning](https://mlsanity.com/paper/1805.11724/) (2018)
- [Material Recognition in the Wild with the Materials in Context Database](https://mlsanity.com/paper/1412.0623/) (2014)
- [Deep TEN: Texture Encoding Network](https://mlsanity.com/paper/1612.02844/) (2016)
- [From BoW to CNN: Two Decades of Texture Representation for Texture Classification](https://mlsanity.com/paper/1801.10324/) (2018)
- [Deep Texture Manifold for Ground Terrain Recognition](https://mlsanity.com/paper/1803.10896/) (2018)
- [Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding](https://mlsanity.com/paper/1808.04505/) (2018)
- [M2D2: A Massively Multi-domain Language Modeling Dataset](https://mlsanity.com/paper/2210.07370/) (2022)
- [Exploring Hierarchical Graph Representation for Large-Scale Zero-Shot Image Classification](https://mlsanity.com/paper/2203.01386/) (2022)
- [HGCLIP: Exploring Vision-Language Models with Graph Representations for Hierarchical Understanding](https://mlsanity.com/paper/2311.14064/) (2023)
- [Learning Hierarchical Graph Neural Networks for Image Clustering](https://mlsanity.com/paper/2107.01319/) (2021)
- [Zero-Shot Learning with Common Sense Knowledge Graphs](https://mlsanity.com/paper/2006.10713/) (2020)
- [Prototype Propagation Networks (PPN) for Weakly-supervised Few-shot Learning on Category Graph](https://mlsanity.com/paper/1905.04042/) (2019)
- [Recognizing Material Properties from Images](https://mlsanity.com/paper/1801.03127/) (2018)
- [RADAM: Texture Recognition through Randomized Aggregated Encoding of Deep Activation Maps](https://mlsanity.com/paper/2303.04554/) (2023)
- [Differential Angular Imaging for Material Recognition](https://mlsanity.com/paper/1612.02372/) (2016)
- [Deep Hierarchical Classification for Category Prediction in E-commerce System](https://mlsanity.com/paper/2005.06692/) (2020)
- [Differential Viewpoints for Ground Terrain Material Recognition](https://mlsanity.com/paper/2009.11072/) (2020)
- [Material Recognition from Local Appearance in Global Context](https://mlsanity.com/paper/1611.09394/) (2016)

## Built on by (most cited first, 3 of 3)

- [New York Smells: A Large Multimodal Dataset for Olfaction](https://mlsanity.com/paper/2511.20544/) (2025)
- [Minimizing Hyperbolic Embedding Distortion with LLM-Guided Hierarchy Restructuring](https://mlsanity.com/paper/2511.20679/) (2025)
- [Minimalist Visual Inertial Odometry](https://mlsanity.com/paper/2605.19990/) (2026)

## Abstract (arXiv)

We introduce a taxonomy of materials for hierarchical recognition from local appearance. Our taxonomy is motivated by vision applications and is arranged according to the physical traits of materials. We contribute a diverse, in-the-wild dataset with images and depth maps of the taxonomy classes. Utilizing the taxonomy and dataset, we present a method for hierarchical material recognition based on graph attention networks. Our model leverages the taxonomic proximity between classes and achieves state-of-the-art performance. We demonstrate the model's potential to generalize to adverse, real-world imaging conditions, and that novel views rendered using the depth maps can enhance this capability. Finally, we show the model's capacity to rapidly learn new materials in a few-shot learning setting.