mlsanityData to 14 Oct 2026

Optimizing rgb-d semantic segmentation through multi-modal interaction and pooling attention

arXiv 2311.11312 ↗First posted 19 Nov 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Semantic segmentation of RGB-D images involves understanding the appearance and spatial relationships of objects within a scene, which requires careful consideration of various factors. However, in indoor environments, the simple input of RGB and depth images often results in a relatively limited acquisition of semantic and spatial information, leading to suboptimal segmentation outcomes. To address this, we propose the Multi-modal Interaction and Pooling Attention Network (MIPANet), a novel approach designed to harness the interactive synergy between RGB and depth modalities, optimizing the utilization of complementary information. Specifically, we incorporate a Multi-modal Interaction Fusion Module (MIM) into the deepest layers of the network. This module is engineered to facilitate the fusion of RGB and depth information, allowing for mutual enhancement and correction. Additionally, we introduce a Pooling Attention Module (PAM) at various stages of the encoder. This module serves to amplify the features extracted by the network and integrates the module's output into the decoder in a targeted manner, significantly improving semantic segmentation performance. Our experimental results demonstrate that MIPANet outperforms existing methods on two indoor scene datasets, NYUDv2 and SUN-RGBD, underscoring its effectiveness in enhancing RGB-D semantic segmentation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 75 references that are in the map. First posted on arXiv 19 Nov 2023.

Builds on

32 of 75 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  5. 2017Squeeze-and-Excitation Networkscited by 4,789
  6. 2018CBAM: Convolutional Block Attention Modulecited by 2,477
  7. 2018Dual Attention Network for Scene Segmentationcited by 1,003
  8. 2019ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networkscited by 575
  9. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  10. 2017Residual Attention Network for Image Classificationcited by 386
  11. 2014Learning Rich Features from RGB-D Images for Object Detection and Segmentationcited by 127
  12. 2020Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentationcited by 122
20 more
  1. 2019ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentationcited by 107
  2. 2014Deep Convolutional Neural Fields for Depth Estimation from a Single Imagecited by 106
  3. 2020Efficient RGB-D Semantic Segmentation for Indoor Scene Analysiscited by 84
  4. 2021ShapeConv: Shape-aware Convolutional Layer for Indoor RGB-D Semantic Segmentationcited by 77
  5. 2020Siamese Network for RGB-D Salient Object Detection and Beyondcited by 67
  6. 2018RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentationcited by 66
  7. 2021EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentationcited by 59
  8. 2013Indoor Semantic Segmentation using depth informationcited by 56
  9. 2022Dynamic Multimodal Fusioncited by 55
  10. 2021GiT: Graph Interactive Transformer for Vehicle Re-identificationcited by 49
  11. 2020Spatial Information Guided Convolution for Real-Time RGBD Semantic Segmentationcited by 48
  12. 2020Exploring Spatial Significance via Hybrid Pyramidal Graph Network for Vehicle Re-identificationcited by 37
  13. 2022Efficient Multi-Task RGB-D Scene Analysis for Indoor Environmentscited by 33
  14. 2020Polarization-driven Semantic Segmentation via Efficient Attention-bridged Fusioncited by 24
  15. 2023Pixel Difference Convolutional Network for RGB-D Semantic Segmentationcited by 13
  16. 2022HSGNet: Object Re-identification with Hierarchical Similarity Graph Networkcited by 6
  17. 2021A Competitive Method to VIPriors Object Detection Challengecited by 5
  18. 2023Spatial-information Guided Adaptive Context-aware Network for Efficient RGB-D Semantic Segmentationcited by 4
  19. 2020Attention-guided Chained Context Aggregation for Semantic Segmentationcited by 3
  20. 2022A Competitive Method for Dog Nose-print Re-identificationcited by 3

Built on by

Not cited yet.