# Middle-level Fusion for Lightweight RGB-D Salient Object Detection

arXiv: https://arxiv.org/abs/2104.11543 · HTML: https://mlsanity.com/paper/2104.11543/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 89 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [EGNet:Edge Guidance Network for Salient Object Detection](https://mlsanity.com/paper/1908.08297/) (2019)
- [A Simple Pooling-Based Design for Real-Time Salient Object Detection](https://mlsanity.com/paper/1904.09569/) (2019)
- [Multi-scale Interactive Network for Salient Object Detection](https://mlsanity.com/paper/2007.09062/) (2020)
- [Deeply supervised salient object detection with short connections](https://mlsanity.com/paper/1611.04849/) (2016)
- [Salient Object Detection in the Deep Learning Era: An In-Depth Survey](https://mlsanity.com/paper/1904.09146/) (2019)
- [Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks](https://mlsanity.com/paper/1907.06781/) (2019)
- [Salient Object Detection: A Survey](https://mlsanity.com/paper/1411.5878/) (2014)
- [Highly Efficient Salient Object Detection with 100K Parameters](https://mlsanity.com/paper/2003.05643/) (2020)
- [JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection](https://mlsanity.com/paper/2004.08515/) (2020)
- [RGB-D Salient Object Detection: A Survey](https://mlsanity.com/paper/2008.00230/) (2020)
- [A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06811/) (2020)
- [Uncertainty Inspired RGB-D Saliency Detection](https://mlsanity.com/paper/2009.03075/) (2020)
- [Accurate RGB-D Salient Object Detection via Collaborative Learning](https://mlsanity.com/paper/2007.11782/) (2020)
- [RGB-D Salient Object Detection with Cross-Modality Modulation and Selection](https://mlsanity.com/paper/2007.07051/) (2020)
- [Cross-Modal Weighting Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.04901/) (2020)
- [RGBD Salient Object Detection via Deep Fusion](https://mlsanity.com/paper/1607.03333/) (2016)
- [Bilateral Attention Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2004.14582/) (2020)
- [MobileSal: Extremely Efficient RGB-D Salient Object Detection](https://mlsanity.com/paper/2012.13095/) (2020)
- [DNA: Deeply-supervised Nonlinear Aggregation for Salient Object Detection](https://mlsanity.com/paper/1903.12476/) (2019)
- [Few-Cost Salient Object Detection with Adversarial-Paced Learning](https://mlsanity.com/paper/2104.01928/) (2021)
- [Edge-guided Non-local Fully Convolutional Network for Salient Object Detection](https://mlsanity.com/paper/1908.02460/) (2019)
- [Depthwise Non-local Module for Fast Salient Object Detection Using a Single Thread](https://mlsanity.com/paper/2001.08057/) (2020)
- [Edge Preserving and Multi-Scale Contextual Neural Network for Salient Object Detection](https://mlsanity.com/paper/1608.08029/) (2016)

## Built on by (most cited first, 5 of 5)

- [Point-aware Interaction and CNN-induced Refinement Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2308.08930/) (2023)
- [CalibNet: Dual-branch Cross-modal Calibration for RGB-D Salient Instance Segmentation](https://mlsanity.com/paper/2307.08098/) (2023)
- [Salient Object Detection From Arbitrary Modalities](https://mlsanity.com/paper/2405.03352/) (2024)
- [CSDNet: Detect Salient Object in Depth-Thermal via A Lightweight Cross Shallow and Deep Perception Network](https://mlsanity.com/paper/2403.10104/) (2024)
- [RSONet: Region-guided Selective Optimization Network for RGB-T Salient Object Detection](https://mlsanity.com/paper/2603.12685/) (2026)

## Abstract (arXiv)

Most existing lightweight RGB-D salient object detection (SOD) models are based on two-stream structure or single-stream structure. The former one first uses two sub-networks to extract unimodal features from RGB and depth images, respectively, and then fuses them for SOD. While, the latter one directly extracts multi-modal features from the input RGB-D images and then focuses on exploiting cross-level complementary information. However, two-stream structure based models inevitably require more parameters and single-stream structure based ones cannot well exploit the cross-modal complementary information since they ignore the modality difference. To address these issues, we propose to employ the middle-level fusion structure for designing lightweight RGB-D SOD model in this paper, which first employs two sub-networks to extract low- and middle-level unimodal features, respectively, and then fuses those extracted middle-level unimodal features for extracting corresponding high-level multi-modal features in the subsequent sub-network. Different from existing models, this structure can effectively exploit the cross-modal complementary information and significantly reduce the network's parameters, simultaneously. Therefore, a novel lightweight SOD model is designed, which contains a information-aware multi-modal feature fusion (IMFF) module for effectively capturing the cross-modal complementary information and a lightweight feature-level and decision-level feature fusion (LFDF) module for aggregating the feature-level and the decision-level saliency information in different stages with less parameters. Our proposed model has only 3.9M parameters and runs at 33 FPS. The experimental results on several benchmark datasets verify the effectiveness and superiority of the proposed method over some state-of-the-art methods.