# A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection

arXiv: https://arxiv.org/abs/2007.06811 · HTML: https://mlsanity.com/paper/2007.06811/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jul 2020.

## Built on by (most cited first, 30 of 67)

- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection](https://mlsanity.com/paper/2203.02688/) (2022)
- [Specificity-preserving RGB-D Saliency Detection](https://mlsanity.com/paper/2108.08162/) (2021)
- [SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection](https://mlsanity.com/paper/2204.05585/) (2022)
- [Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion](https://mlsanity.com/paper/2103.11832/) (2021)
- [RGB-D Saliency Detection via Cascaded Mutual Information Minimization](https://mlsanity.com/paper/2109.07246/) (2021)
- [ZoomNeXt: A Unified Collaborative Pyramid Network for Camouflaged Object Detection](https://mlsanity.com/paper/2310.20208/) (2023)
- [CAVER: Cross-Modal View-Mixed Transformer for Bi-Modal Salient Object Detection](https://mlsanity.com/paper/2112.02363/) (2021)
- [TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Network](https://mlsanity.com/paper/2108.03990/) (2021)
- [GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformer](https://mlsanity.com/paper/2406.01210/) (2024)
- [CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detection](https://mlsanity.com/paper/2210.02843/) (2022)
- [RGB-D Salient Object Detection via 3D Convolutional Neural Networks](https://mlsanity.com/paper/2101.10241/) (2021)
- [Cross-modality Discrepant Interaction Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2108.01971/) (2021)
- [Depth Quality-Inspired Feature Manipulation for Efficient RGB-D Salient Object Detection](https://mlsanity.com/paper/2107.01779/) (2021)
- [HiDAnet: RGB-D Salient Object Detection via Hierarchical Depth Awareness](https://mlsanity.com/paper/2301.07405/) (2023)
- [VST++: Efficient and Stronger Visual Saliency Transformer](https://mlsanity.com/paper/2310.11725/) (2023)
- [Glass Segmentation with RGB-Thermal Image Pairs](https://mlsanity.com/paper/2204.05453/) (2022)
- [Fantastic Animals and Where to Find Them: Segment Any Marine Animal with Dual SAM](https://mlsanity.com/paper/2404.04996/) (2024)
- [Context-Aware Interaction Network for RGB-T Semantic Segmentation](https://mlsanity.com/paper/2401.01624/) (2024)
- [Self-Supervised Pretraining for RGB-D Salient Object Detection](https://mlsanity.com/paper/2101.12482/) (2021)
- [Multi-Source Fusion and Automatic Predictor Selection for Zero-Shot Video Object Segmentation](https://mlsanity.com/paper/2108.05076/) (2021)
- [Spider: A Unified Framework for Context-dependent Concept Segmentation](https://mlsanity.com/paper/2405.01002/) (2024)
- [Towards Diverse Binary Segmentation via A Simple yet General Gated Network](https://mlsanity.com/paper/2303.10396/) (2023)
- [BTS-Net: Bi-directional Transfer-and-Selection Network For RGB-D Salient Object Detection](https://mlsanity.com/paper/2104.01784/) (2021)
- [Open-Vocabulary Camouflaged Object Segmentation](https://mlsanity.com/paper/2311.11241/) (2023)
- [M2SNet: Multi-scale in Multi-scale Subtraction Network for Medical Image Segmentation](https://mlsanity.com/paper/2303.10894/) (2023)
- [Joint Learning of Salient Object Detection, Depth Estimation and Contour Extraction](https://mlsanity.com/paper/2203.04895/) (2022)
- [Promoting Saliency From Depth: Deep Unsupervised RGB-D Saliency Detection](https://mlsanity.com/paper/2205.07179/) (2022)
- [Robust RGB-D Fusion for Saliency Detection](https://mlsanity.com/paper/2208.01762/) (2022)
- [Symmetry-Aware Transformer-based Mirror Detection](https://mlsanity.com/paper/2207.06332/) (2022)

## Abstract (arXiv)

Existing RGB-D salient object detection (SOD) approaches concentrate on the cross-modal fusion between the RGB stream and the depth stream. They do not deeply explore the effect of the depth map itself. In this work, we design a single stream network to directly use the depth map to guide early fusion and middle fusion between RGB and depth, which saves the feature encoder of the depth stream and achieves a lightweight and real-time model. We tactfully utilize depth information from two perspectives: (1) Overcoming the incompatibility problem caused by the great difference between modalities, we build a single stream encoder to achieve the early fusion, which can take full advantage of ImageNet pre-trained backbone model to extract rich and discriminative features. (2) We design a novel depth-enhanced dual attention module (DEDA) to efficiently provide the fore-/back-ground branches with the spatially filtered features, which enables the decoder to optimally perform the middle fusion. Besides, we put forward a pyramidally attended feature extraction module (PAFE) to accurately localize the objects of different scales. Extensive experiments demonstrate that the proposed model performs favorably against most state-of-the-art methods under different evaluation metrics. Furthermore, this model is 55.5% lighter than the current lightest model and runs at a real-time speed of 32 FPS when processing a 384 × 384 image.