# Material Recognition in the Wild with the Materials in Context Database

arXiv: https://arxiv.org/abs/1412.0623 · HTML: https://mlsanity.com/paper/1412.0623/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2014.

## Built on by (most cited first, 30 of 79)

- [How Far is Video Generation from World Model: A Physical Law Perspective](https://mlsanity.com/paper/2411.02385/) (2024)
- [MSeg: A Composite Dataset for Multi-domain Semantic Segmentation](https://mlsanity.com/paper/2112.13762/) (2021)
- [Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models](https://mlsanity.com/paper/2312.08962/) (2023)
- [Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracing](https://mlsanity.com/paper/2211.03017/) (2022)
- [Physical Property Understanding from Language-Embedded Feature Fields](https://mlsanity.com/paper/2404.04242/) (2024)
- [Efficient Physics Simulation for 3D Scenes via MLLM-Guided Gaussian Splatting](https://mlsanity.com/paper/2411.12789/) (2024)
- [Materialistic: Selecting Similar Materials in Images](https://mlsanity.com/paper/2305.13291/) (2023)
- [Beyond Image to Depth: Improving Depth Prediction using Echoes](https://mlsanity.com/paper/2103.08468/) (2021)
- [What a MESS: Multi-Domain Evaluation of Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/2306.15521/) (2023)
- [ZeST: Zero-Shot Material Transfer from a Single Image](https://mlsanity.com/paper/2404.06425/) (2024)
- [Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixels](https://mlsanity.com/paper/2508.17437/) (2025)
- [A Dense Material Segmentation Dataset for Indoor and Outdoor Scene Parsing](https://mlsanity.com/paper/2207.10614/) (2022)
- [PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes](https://mlsanity.com/paper/2207.00757/) (2022)
- [Crafting Training Degradation Distribution for the Accuracy-Generalization Trade-off in Real-World Super-Resolution](https://mlsanity.com/paper/2305.18107/) (2023)
- [A Survey on Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2302.09899/) (2023)
- [Context-self contrastive pretraining for crop type semantic segmentation](https://mlsanity.com/paper/2104.04310/) (2021)
- [Symmetry-Aware Transformer-based Mirror Detection](https://mlsanity.com/paper/2207.06332/) (2022)
- [Dual-Scale Single Image Dehazing Via Neural Augmentation](https://mlsanity.com/paper/2209.05913/) (2022)
- [SAMa: Material-aware 3D Selection and Segmentation](https://mlsanity.com/paper/2411.19322/) (2024)
- [It Is All About Data: A Survey on the Effects of Data on Adversarial Robustness](https://mlsanity.com/paper/2303.09767/) (2023)
- [MARBLE: Material Recomposition and Blending in CLIP-Space](https://mlsanity.com/paper/2506.05313/) (2025)
- [Measuring the Interpretability of Unsupervised Representations via Quantized Reverse Probing](https://mlsanity.com/paper/2209.03268/) (2022)
- [MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assets](https://mlsanity.com/paper/2404.13923/) (2024)
- [The joint role of geometry and illumination on material recognition](https://mlsanity.com/paper/2101.02496/) (2021)
- [Visual Vibration Tomography: Estimating Interior Material Properties from Monocular Video](https://mlsanity.com/paper/2104.02735/) (2021)
- [Photo-to-Shape Material Transfer for Diverse Structures](https://mlsanity.com/paper/2205.04018/) (2022)
- [One-shot recognition of any material anywhere using contrastive learning with physics-based rendering](https://mlsanity.com/paper/2212.00648/) (2022)
- [Beyond Visual Field of View: Perceiving 3D Environment with Echoes and Vision](https://mlsanity.com/paper/2207.01136/) (2022)
- [MERLiN: Single-Shot Material Estimation and Relighting for Photometric Stereo](https://mlsanity.com/paper/2409.00674/) (2024)
- [Leveraging Bitstream Metadata for Fast, Accurate, Generalized Compressed Video Quality Enhancement](https://mlsanity.com/paper/2202.00011/) (2022)

## Abstract (arXiv)

Recognizing materials in real-world images is a challenging task. Real-world materials have rich surface texture, geometry, lighting conditions, and clutter, which combine to make the problem particularly difficult. In this paper, we introduce a new, large-scale, open dataset of materials in the wild, the Materials in Context Database (MINC), and combine this dataset with deep learning to achieve material recognition and segmentation of images in the wild. MINC is an order of magnitude larger than previous material databases, while being more diverse and well-sampled across its 23 categories. Using MINC, we train convolutional neural networks (CNNs) for two tasks: classifying materials from patches, and simultaneous material recognition and segmentation in full images. For patch-based classification on MINC we found that the best performing CNN architectures can achieve 85.2% mean class accuracy. We convert these trained CNN classifiers into an efficient fully convolutional framework combined with a fully connected conditional random field (CRF) to predict the material at every pixel in an image, achieving 73.1% mean class accuracy. Our experiments demonstrate that having a large, well-sampled dataset such as MINC is crucial for real-world material recognition and segmentation.