# Geometry-Informed Material Recognition

arXiv: https://arxiv.org/abs/1607.05338 · HTML: https://mlsanity.com/paper/1607.05338/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2016.

## Built on by (most cited first, 5 of 5)

- [Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks](https://mlsanity.com/paper/2112.01715/) (2021)
- [PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility](https://mlsanity.com/paper/2108.08943/) (2021)
- [DPF: Learning Dense Prediction Fields with Weak Supervision](https://mlsanity.com/paper/2303.16890/) (2023)
- [Object-Oriented Material Classification and 3D Clustering for Improved Semantic Perception and Mapping in Mobile Robots](https://mlsanity.com/paper/2407.06077/) (2024)
- [Construction material classification on imbalanced datasets using Vision Transformer (ViT) architecture](https://mlsanity.com/paper/2108.09527/) (2021)

## Abstract (arXiv)

Our goal is to recognize material categories using images and geometry information. In many applications, such as construction management, coarse geometry information is available. We investigate how 3D geometry (surface normals, camera intrinsic and extrinsic parameters) can be used with 2D features (texture and color) to improve material classification. We introduce a new dataset, GeoMat, which is the first to provide both image and geometry data in the form of: (i) training and testing patches that were extracted at different scales and perspectives from real world examples of each material category, and (ii) a large scale construction site scene that includes 160 images and over 800,000 hand labeled 3D points. Our results show that using 2D and 3D features both jointly and independently to model materials improves classification accuracy across multiple scales and viewing directions for both material patches and images of a large scale construction site scene.