# MatPredict: a dataset and benchmark for learning material properties of diverse indoor objects

arXiv: https://arxiv.org/abs/2505.13201 · HTML: https://mlsanity.com/paper/2505.13201/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 19 May 2025.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Neural Inverse Rendering of an Indoor Scene from a Single Image](https://mlsanity.com/paper/1901.02453/) (2019)
- [CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering](https://mlsanity.com/paper/1808.08601/) (2018)
- [PCANet: A Simple Deep Learning Baseline for Image Classification?](https://mlsanity.com/paper/1404.3606/) (2014)
- [MatSynth: A Modern PBR Materials Dataset](https://mlsanity.com/paper/2401.06056/) (2024)
- [Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks](https://mlsanity.com/paper/1612.07429/) (2016)
- [A Similarity Measure for Material Appearance](https://mlsanity.com/paper/1905.01562/) (2019)
- [Recognizing Material Properties from Images](https://mlsanity.com/paper/1801.03127/) (2018)
- [Deep Learning for Material recognition: most recent advances and open challenges](https://mlsanity.com/paper/2012.07495/) (2020)

## Built on by (most cited first, 2 of 2)

- [Adaptive Volumetric Mechanical Property Fields Invariant to Resolution](https://mlsanity.com/paper/2606.18231/) (2026)
- [WCM: World-Cognition Model for Generalizable Human-Robot Interaction](https://mlsanity.com/paper/2607.22999/) (2026)

## Abstract (arXiv)

Determining material properties from camera images can expand the ability to identify complex objects in indoor environments, which is valuable for consumer robotics applications. To support this, we introduce MatPredict, a dataset that combines the high-quality synthetic objects from Replica dataset with MatSynth dataset's material properties classes - to create objects with diverse material properties. We select 3D meshes of specific foreground objects and render them with different material properties. In total, we generate 18 commonly occurring objects with 14 different materials. We showcase how we provide variability in terms of lighting and camera placement for these objects. Next, we provide a benchmark for inferring material properties from visual images using these perturbed models in the scene, discussing the specific neural network models involved and their performance based on different image comparison metrics. By accurately simulating light interactions with different materials, we can enhance realism, which is crucial for training models effectively through large-scale simulations. This research aims to revolutionize perception in consumer robotics. The dataset is provided hrefhttps://huggingface.co/datasets/UMTRI/MatPredicthere and the code is provided hrefhttps://github.com/arpan-kusari/MatPredicthere.