RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentation
Not on the map. Why?
Abstract (arXiv)
Indoor semantic segmentation has always been a difficult task in computer vision. In this paper, we propose an RGB-D residual encoder-decoder architecture, named RedNet, for indoor RGB-D semantic segmentation. In RedNet, the residual module is applied to both the encoder and decoder as the basic building block, and the skip-connection is used to bypass the spatial feature between the encoder and decoder. In order to incorporate the depth information of the scene, a fusion structure is constructed, which makes inference on RGB image and depth image separately, and fuses their features over several layers. In order to efficiently optimize the network's parameters, we propose a `pyramid supervision' training scheme, which applies supervised learning over different layers in the decoder, to cope with the problem of gradients vanishing. Experiment results show that the proposed RedNet(ResNet-50) achieves a state-of-the-art mIoU accuracy of 47.8% on the SUN RGB-D benchmark dataset.
Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Jun 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 8; 2022: 20; 2023: 14; 2024: 11; 2025: 7; 2026: 5.
65 papers.
Sort:
By area:
- 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 151
- 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
- 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
- 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
- 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
- 2022Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)cited by 115
- 2022Habitat-Matterport 3D Semantics Datasetcited by 109
- 2022Navigating to Objects in the Real Worldcited by 104
- 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
- 2021Auxiliary Tasks and Exploration Enable ObjectNavcited by 73
- 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 66
- 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learningcited by 60
38 more
- 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
- 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identificationcited by 46
- 2025World-in-World: World Models in a Closed-Loop Worldcited by 44
- 2021Improving Multi-Modal Learning with Uni-Modal Teacherscited by 40
- 2023Learning Navigational Visual Representations with Semantic Map Supervisioncited by 40
- 2022Iterative Vision-and-Language Navigationcited by 34
- 2022Simple and Effective Synthesis of Indoor 3D Scenescited by 33
- 2022Episodic Memory Question Answeringcited by 32
- 2022Stubborn: A Strong Baseline for Indoor Object Navigationcited by 31
- 2022WayFAST: Navigation with Predictive Traversability in the Fieldcited by 30
- 2023Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentationcited by 24
- 2022PEANUT: Predicting and Navigating to Unseen Targetscited by 22
- 2024OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representationcited by 17
- 2023AsymFormer: Asymmetrical Cross-Modal Representation Learning for Mobile Platform Real-Time RGB-D Semantic Segmentationcited by 16
- 2024Enhancing Multi-Robot Semantic Navigation Through Multimodal Chain-of-Thought Score Collaborationcited by 14
- 2023Pixel Difference Convolutional Network for RGB-D Semantic Segmentationcited by 13
- 2023RGB-D And Thermal Sensor Fusion: A Systematic Literature Reviewcited by 12
- 2023LaMPP: Language Models as Probabilistic Priors for Perception and Actioncited by 11
- 2022Online Learning of Reusable Abstract Models for Object Goal Navigationcited by 9
- 2022DCANet: Differential Convolution Attention Network for RGB-D Semantic Segmentationcited by 9
- 2023Exploitation-Guided Exploration for Semantic Embodied Navigationcited by 9
- 2023Joint Depth Prediction and Semantic Segmentation with Multi-View SAMcited by 8
- 2023Efficient Multi-Task Scene Analysis with RGB-D Transformerscited by 7
- 2022Attention-based Dual Supervised Decoder for RGBD Semantic Segmentationcited by 6
- 2024Perception Matters: Enhancing Embodied AI with Uncertainty-Aware Semantic Segmentationcited by 6
- 2025NAF: Zero-Shot Feature Upsampling via Neighborhood Attention Filteringcited by 6
- 2021Deep feature selection-and-fusion for RGB-D semantic segmentationcited by 5
- 2023GAMUS: A Geometry-aware Multi-modal Semantic Segmentation Benchmark for Remote Sensing Datacited by 4
- 2025Revisiting Cross-Modal Knowledge Distillation: A Disentanglement Approach for RGBD Semantic Segmentationcited by 4
- 2021Global-Local Propagation Network for RGB-D Semantic Segmentationcited by 3
- 2021Review on Indoor RGB-D Semantic Segmentation with Deep Convolutional Neural Networkscited by 3
- 2023Implicit Obstacle Map-driven Indoor Navigation Model for Robust Obstacle Avoidancecited by 3
- 2025MVIP -- A Dataset and Methods for Application Oriented Multi-View and Multi-Modal Industrial Part Recognitioncited by 3
- 2025Distilling LLM Prior to Flow Model for Generalizable Agent's Imagination in Object Goal Navigationcited by 3
- 2022Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformerscited by 2
- 2024BenchCloudVision: A Benchmark Analysis of Deep Learning Approaches for Cloud Detection and Segmentation in Remote Sensing Imagerycited by 2
- 2024Leveraging Large Language Model-based Room-Object Relationships Knowledge for Enhancing Multimodal-Input Object Goal Navigationcited by 2
- 2024Depth-guided Texture Diffusion for Image Semantic Segmentationcited by 2
50 most cited of 65 All 65 (JSON)
- 2026Multi-Modal Semantic Segmentation of Electrolyzer Components for Sustainable Hydrogen Technologies: A Dual-Branch Deep Learning Approach17 Jul 2026
- 2026Plug-and-Play Label Map Diffusion for Universal Goal-Oriented Navigation7 May 2026
- 2026DeepSight: Bridging Depth Maps and Language with a Depth-Driven Multimodal Model6 Mar 2026
- 2026Sim2Real SAR Image Restoration: Metadata-Driven Models for Joint Despeckling and Sidelobes Reduction4 Jan 2026
- 2026Efficient Prediction of Dense Visual Embeddings via Distillation and RGB-D Transformers1 Jan 2026
- 2025Power of Boundary and Reflection: Semantic Transparent Object Segmentation using Pyramid Vision Transformer with Transparent Cues7 Dec 2025
- 2025NAF: Zero-Shot Feature Upsampling via Neighborhood Attention Filtering23 Nov 2025
- 2025World-in-World: World Models in a Closed-Loop World20 Oct 2025
- 2025Distilling LLM Prior to Flow Model for Generalizable Agent's Imagination in Object Goal Navigation13 Aug 2025
- 2025Revisiting Cross-Modal Knowledge Distillation: A Disentanglement Approach for RGBD Semantic Segmentation30 May 2025
- 2025A Large Vision-Language Model based Environment Perception System for Visually Impaired People25 Apr 2025
- 2025MVIP -- A Dataset and Methods for Application Oriented Multi-View and Multi-Modal Industrial Part Recognition21 Feb 2025
38 more
- 2024Enhancing Multi-Robot Semantic Navigation Through Multimodal Chain-of-Thought Score Collaboration24 Dec 2024
- 2024Diffusion-based RGB-D Semantic Segmentation with Deformable Attention Transformer23 Sep 2024
- 2024Depth-guided Texture Diffusion for Image Semantic Segmentation17 Aug 2024
- 2024Perception Matters: Enhancing Embodied AI with Uncertainty-Aware Semantic Segmentation5 Aug 2024
- 2024CSFNet: A Cosine Similarity Fusion Network for Real-Time RGB-X Semantic Segmentation of Driving Scenes1 Jul 2024
- 2024OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representation26 Mar 2024
- 2024Leveraging Large Language Model-based Room-Object Relationships Knowledge for Enhancing Multimodal-Input Object Goal Navigation21 Mar 2024
- 2024BenchCloudVision: A Benchmark Analysis of Deep Learning Approaches for Cloud Detection and Segmentation in Remote Sensing Imagery21 Feb 2024
- 2024RFI-DRUnet: Restoring dynamic spectra corrupted by radio frequency interference -- Application to pulsar observations21 Feb 2024
- 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Models16 Feb 2024
- 2024Semantic-Syntactic Discrepancy in Images (SSDI): Learning Meaning and Order of Features from Natural Images31 Jan 2024
- 2023Optimizing rgb-d semantic segmentation through multi-modal interaction and pooling attention19 Nov 2023
- 2023Exploitation-Guided Exploration for Semantic Embodied Navigation6 Nov 2023
- 2023Joint Depth Prediction and Semantic Segmentation with Multi-View SAM31 Oct 2023
- 2023AsymFormer: Asymmetrical Cross-Modal Representation Learning for Mobile Platform Real-Time RGB-D Semantic Segmentation25 Sep 2023
- 2023Implicit Obstacle Map-driven Indoor Navigation Model for Robust Obstacle Avoidance24 Aug 2023
- 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation14 Aug 2023
- 2023Learning Navigational Visual Representations with Semantic Map Supervision23 Jul 2023
- 2023A Self-Supervised Miniature One-Shot Texture Segmentation (MOSTS) Model for Real-Time Robot Navigation and Embedded Applications15 Jun 2023
- 2023Efficient Multi-Task Scene Analysis with RGB-D Transformers8 Jun 2023
- 2023GAMUS: A Geometry-aware Multi-modal Semantic Segmentation Benchmark for Remote Sensing Data24 May 2023
- 2023RGB-D And Thermal Sensor Fusion: A Systematic Literature Review19 May 2023
- 2023Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentation21 Apr 2023
- 2023Pixel Difference Convolutional Network for RGB-D Semantic Segmentation23 Feb 2023
- 2023LaMPP: Language Models as Probabilistic Priors for Perception and Action3 Feb 2023
- 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigation8 Dec 2022
- 2022PEANUT: Predicting and Navigating to Unseen Targets5 Dec 2022
- 2022Navigating to Objects in the Real World2 Dec 2022
- 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identification1 Dec 2022
- 2022DCANet: Differential Convolution Attention Network for RGB-D Semantic Segmentation13 Oct 2022
- 2022Habitat-Matterport 3D Semantics Dataset11 Oct 2022
- 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learning6 Oct 2022
- 2022Iterative Vision-and-Language Navigation6 Oct 2022
- 2022EraseNet: A Recurrent Residual Network for Supervised Document Cleaning3 Oct 2022
- 2022Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers13 Jul 2022
- 2022Housekeep: Tidying Virtual Households using Commonsense Reasoning22 May 2022
- 2022Episodic Memory Question Answering3 May 2022
- 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale7 Apr 2022
50 newest of 65 All 65 (JSON)
- 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
- 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
- 2025World-in-World: World Models in a Closed-Loop Worldcited by 44
- 2024OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representationcited by 17
- 2025World-in-World: World Models in a Closed-Loop World20 Oct 2025
- 2024OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representation26 Mar 2024
- 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation14 Aug 2023
- 2021Pathdreamer: A World Model for Indoor Navigation18 May 2021
- 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 66
- 2025A Large Vision-Language Model based Environment Perception System for Visually Impaired Peoplecited by 1
- 2026DeepSight: Bridging Depth Maps and Language with a Depth-Driven Multimodal Modelcited by 0
- 2026DeepSight: Bridging Depth Maps and Language with a Depth-Driven Multimodal Model6 Mar 2026
- 2025A Large Vision-Language Model based Environment Perception System for Visually Impaired People25 Apr 2025
- 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Models16 Feb 2024
- 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 151
- 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
- 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
- 2022Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)cited by 115
- 2022Habitat-Matterport 3D Semantics Datasetcited by 109
- 2022Navigating to Objects in the Real Worldcited by 104
- 2021Auxiliary Tasks and Exploration Enable ObjectNavcited by 73
- 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identificationcited by 46
- 2021Improving Multi-Modal Learning with Uni-Modal Teacherscited by 40
- 2023Learning Navigational Visual Representations with Semantic Map Supervisioncited by 40
- 2022Iterative Vision-and-Language Navigationcited by 34
- 2022Simple and Effective Synthesis of Indoor 3D Scenescited by 33
12 most cited of 52 All 52 (JSON)
- 2026Multi-Modal Semantic Segmentation of Electrolyzer Components for Sustainable Hydrogen Technologies: A Dual-Branch Deep Learning Approach17 Jul 2026
- 2026Sim2Real SAR Image Restoration: Metadata-Driven Models for Joint Despeckling and Sidelobes Reduction4 Jan 2026
- 2026Efficient Prediction of Dense Visual Embeddings via Distillation and RGB-D Transformers1 Jan 2026
- 2025Power of Boundary and Reflection: Semantic Transparent Object Segmentation using Pyramid Vision Transformer with Transparent Cues7 Dec 2025
- 2025NAF: Zero-Shot Feature Upsampling via Neighborhood Attention Filtering23 Nov 2025
- 2025Revisiting Cross-Modal Knowledge Distillation: A Disentanglement Approach for RGBD Semantic Segmentation30 May 2025
- 2025MVIP -- A Dataset and Methods for Application Oriented Multi-View and Multi-Modal Industrial Part Recognition21 Feb 2025
- 2024Diffusion-based RGB-D Semantic Segmentation with Deformable Attention Transformer23 Sep 2024
- 2024Depth-guided Texture Diffusion for Image Semantic Segmentation17 Aug 2024
- 2024Perception Matters: Enhancing Embodied AI with Uncertainty-Aware Semantic Segmentation5 Aug 2024
- 2024CSFNet: A Cosine Similarity Fusion Network for Real-Time RGB-X Semantic Segmentation of Driving Scenes1 Jul 2024
- 2024Leveraging Large Language Model-based Room-Object Relationships Knowledge for Enhancing Multimodal-Input Object Goal Navigation21 Mar 2024
12 newest of 52 All 52 (JSON)