mlsanityData to 14 Oct 2026

A Dense Material Segmentation Dataset for Indoor and Outdoor Scene Parsing

arXiv 2207.10614 ↗First posted 21 Jul 2022cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

A key algorithm for understanding the world is material segmentation, which assigns a label (metal, glass, etc.) to each pixel. We find that a model trained on existing data underperforms in some settings and propose to address this with a large-scale dataset of 3.2 million dense segments on 44,560 indoor and outdoor images, which is 23x more segments than existing data. Our data covers a more diverse set of scenes, objects, viewpoints and materials, and contains a more fair distribution of skin types. We show that a model trained on our data outperforms a state-of-the-art model across datasets and viewpoints. We propose a large-scale scene parsing benchmark and baseline of 0.729 per-pixel accuracy, 0.585 mean class accuracy and 0.420 mean IoU across 46 materials.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 25 of its 49 references that are in the map. First posted on arXiv 21 Jul 2022.

Builds on

25 of 49 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  5. 2016Pyramid Scene Parsing Networkcited by 2,479
  6. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  7. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,133
  8. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  9. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  10. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
  11. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 500
  12. 2016ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentationcited by 291
13 more
  1. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
  2. 2019ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulationcited by 124
  3. 2014Material Recognition in the Wild with the Materials in Context Databasecited by 79
  4. 2020Fashionpedia: Ontology, Segmentation, and an Attribute Localization Datasetcited by 57
  5. 2019A Dataset of Multi-Illumination Images in the Wildcited by 45
  6. 2018Deep Texture Manifold for Ground Terrain Recognitioncited by 32
  7. 2016A 4D Light-Field Dataset and CNN Architectures for Material Recognitioncited by 26
  8. 2015Deep Learning for Tactile Understanding From Visual and Haptic Datacited by 18
  9. 2018Recognizing Material Properties from Imagescited by 14
  10. 2016Differential Angular Imaging for Material Recognitioncited by 8
  11. 2017A fully end-to-end deep learning approach for real-time simultaneous 3D reconstruction and material recognitioncited by 5
  12. 2020Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer visioncited by 5
  13. 2017Material Recognition CNNs and Hierarchical Planning for Biped Robot Locomotion on Slippery Terraincited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 6; 2025: 4; 2026: 7.

23 papers.

Sort:

By area:

  1. 2023A General Protocol to Probe Large Vision Models for 3D Physical Understandingcited by 32
  2. 2024MaPa: Text-driven Photorealistic Material Painting for 3D Shapescited by 27
  3. 2023Materialistic: Selecting Similar Materials in Imagescited by 26
  4. 2023What a MESS: Multi-Domain Evaluation of Zero-Shot Semantic Segmentationcited by 24
  5. 2024ZeST: Zero-Shot Material Transfer from a Single Imagecited by 24
  6. 2023MMSFormer: Multimodal Transformer for Material and Semantic Segmentationcited by 12
  7. 2024SAMa: Material-aware 3D Selection and Segmentationcited by 10
  8. 2023MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairmentscited by 9
  9. 2025MARBLE: Material Recomposition and Blending in CLIP-Spacecited by 9
  10. 2024MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assetscited by 6
  11. 2022One-shot recognition of any material anywhere using contrastive learning with physics-based renderingcited by 5
  12. 2025Shape and Texture Recognition in Large Vision-Language Modelscited by 3
11 more
  1. 2024Learning Zero-Shot Material States Segmentation, by Implanting Natural Image Patterns in Synthetic Datacited by 2
  2. 2024LatentBKI: Open-Dictionary Continuous Mapping in Visual-Language Latent Spaces with Quantifiable Uncertaintycited by 2
  3. 2026Seeing Through Touch: Tactile-Driven Visual Localization of Material Regionscited by 2
  4. 2025Sonify Anything: Towards Context-Aware Sonic Interactions in ARcited by 1
  5. 2025With Great Context Comes Great Prediction Power: Classifying Objects via Geo-Semantic Scene Graphscited by 0
  6. 2026FMMC: Harnessing the Power of Foundation Models for Accurate Material Classificationcited by 0
  7. 2026Revitalizing Dense Material Segmentation: Stabilized Vision Transformers and the Generalization Paradoxcited by 0
  8. 2026Sub-Semantic Image Segmentationcited by 0
  9. 2026Unified Video Dense Prediction from Disjoint Datacited by 0
  10. 2026BIMScript: Material-Aware Structured Scene Programs for BIM Ingestioncited by 0
  11. 2026Spectral World Renderer: Unified 3D Hyperspectral Unmixingcited by 0