# Deep feature selection-and-fusion for RGB-D semantic segmentation

arXiv: https://arxiv.org/abs/2105.04102 · HTML: https://mlsanity.com/paper/2105.04102/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation](https://mlsanity.com/paper/1906.03525/) (2019)
- [Pyramid Feature Attention Network for Saliency detection](https://mlsanity.com/paper/1903.00179/) (2019)
- [Learning Rich Features from RGB-D Images for Object Detection and Segmentation](https://mlsanity.com/paper/1407.5736/) (2014)
- [Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation](https://mlsanity.com/paper/2007.09183/) (2020)
- [ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation](https://mlsanity.com/paper/1905.10089/) (2019)
- [RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentation](https://mlsanity.com/paper/1806.01054/) (2018)
- [Indoor Semantic Segmentation using depth information](https://mlsanity.com/paper/1301.3572/) (2013)
- [Multi-View Deep Learning for Consistent Semantic Mapping with RGB-D Cameras](https://mlsanity.com/paper/1703.08866/) (2017)
- [Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks](https://mlsanity.com/paper/1608.01082/) (2016)
- [Recurrent Scene Parsing with Perspective Understanding in the Loop](https://mlsanity.com/paper/1705.07238/) (2017)
- [RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic Segmentation](https://mlsanity.com/paper/1907.00135/) (2019)

## Built on by (most cited first, 5 of 5)

- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [CaRe-Ego: Contact-aware Relationship Modeling for Egocentric Interactive Hand-object Segmentation](https://mlsanity.com/paper/2407.05576/) (2024)
- [ComPtr: Towards Diverse Bi-source Dense Prediction Tasks via A Simple yet General Complementary Transformer](https://mlsanity.com/paper/2307.12349/) (2023)
- [MVIP -- A Dataset and Methods for Application Oriented Multi-View and Multi-Modal Industrial Part Recognition](https://mlsanity.com/paper/2502.15448/) (2025)
- [Clothes Grasping and Unfolding Based on RGB-D Semantic Segmentation](https://mlsanity.com/paper/2305.03259/) (2023)

## Abstract (arXiv)

Scene depth information can help visual information for more accurate semantic segmentation. However, how to effectively integrate multi-modality information into representative features is still an open problem. Most of the existing work uses DCNNs to implicitly fuse multi-modality information. But as the network deepens, some critical distinguishing features may be lost, which reduces the segmentation performance. This work proposes a unified and efficient feature selectionand-fusion network (FSFNet), which contains a symmetric cross-modality residual fusion module used for explicit fusion of multi-modality information. Besides, the network includes a detailed feature propagation module, which is used to maintain low-level detailed information during the forward process of the network. Compared with the state-of-the-art methods, experimental evaluations demonstrate that the proposed model achieves competitive performance on two public datasets.