# Exploiting Object-based and Segmentation-based Semantic Features for Deep Learning-based Indoor Scene Classification

arXiv: https://arxiv.org/abs/2404.07739 · HTML: https://mlsanity.com/paper/2404.07739/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 11 Apr 2024.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Semantic-Aware Scene Recognition](https://mlsanity.com/paper/1909.02410/) (2019)
- [Efficient Multi-Task RGB-D Scene Analysis for Indoor Environments](https://mlsanity.com/paper/2207.04526/) (2022)
- [Scene Recognition with Prototype-agnostic Scene Layout](https://mlsanity.com/paper/1909.03234/) (2019)
- [ASK: Adaptively Selecting Key Local Features for RGB-D Scene Recognition](https://mlsanity.com/paper/2110.07703/) (2021)
- [When CNNs Meet Random RNNs: Towards Multi-Level Analysis for RGB-D Object and Scene Recognition](https://mlsanity.com/paper/2004.12349/) (2020)
- [BORM: Bayesian Object Relation Model for Indoor Scene Recognition](https://mlsanity.com/paper/2108.00397/) (2021)
- [Learning Effective RGB-D Representations for Scene Recognition](https://mlsanity.com/paper/1809.06269/) (2018)
- [2D-3D Geometric Fusion Network using Multi-Neighbourhood Graph Convolution for RGB-D Indoor Scene Classification](https://mlsanity.com/paper/2009.11154/) (2020)
- [FOSNet: An End-to-End Trainable Deep Neural Network for Scene Recognition](https://mlsanity.com/paper/1907.07570/) (2019)
- [Attentional Graph Convolutional Network for Structure-aware Audio-Visual Scene Classification](https://mlsanity.com/paper/2301.00145/) (2022)
- [Semantic Clustering for Robust Fine-Grained Scene Recognition](https://mlsanity.com/paper/1607.07614/) (2016)
- [A Deep Learning-based Global and Segmentation-based Semantic Feature Fusion Approach for Indoor Scene Classification](https://mlsanity.com/paper/2302.06432/) (2023)

## Abstract (arXiv)

Indoor scenes are usually characterized by scattered objects and their relationships, which turns the indoor scene classification task into a challenging computer vision task. Despite the significant performance boost in classification tasks achieved in recent years, provided by the use of deep-learning-based methods, limitations such as inter-category ambiguity and intra-category variation have been holding back their performance. To overcome such issues, gathering semantic information has been shown to be a promising source of information towards a more complete and discriminative feature representation of indoor scenes. Therefore, the work described in this paper uses both semantic information, obtained from object detection, and semantic segmentation techniques. While object detection techniques provide the 2D location of objects allowing to obtain spatial distributions between objects, semantic segmentation techniques provide pixel-level information that allows to obtain, at a pixel-level, a spatial distribution and shape-related features of the segmentation categories. Hence, a novel approach that uses a semantic segmentation mask to provide Hu-moments-based segmentation categories' shape characterization, designated by Segmentation-based Hu-Moments Features (SHMFs), is proposed. Moreover, a three-main-branch network, designated by GOS2F2App, that exploits deep-learning-based global features, object-based features, and semantic segmentation-based features is also proposed. GOS2F2App was evaluated in two indoor scene benchmark datasets: SUN RGB-D and NYU Depth V2, where, to the best of our knowledge, state-of-the-art results were achieved on both datasets, which present evidences of the effectiveness of the proposed approach.