# RGB-D Saliency Detection via Cascaded Mutual Information Minimization

arXiv: https://arxiv.org/abs/2109.07246 · HTML: https://mlsanity.com/paper/2109.07246/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 55 references that are in the map.
- First posted on arXiv 15 Sep 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [On Mutual Information Maximization for Representation Learning](https://mlsanity.com/paper/1907.13625/) (2019)
- [F3Net: Fusion, Feedback and Focus for Salient Object Detection](https://mlsanity.com/paper/1911.11445/) (2019)
- [Hierarchical Neural Architecture Search for Deep Stereo Matching](https://mlsanity.com/paper/2010.13501/) (2020)
- [Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks](https://mlsanity.com/paper/1907.06781/) (2019)
- [UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders](https://mlsanity.com/paper/2004.05763/) (2020)
- [Weakly-Supervised Salient Object Detection via Scribble Annotations](https://mlsanity.com/paper/2003.07685/) (2020)
- [JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection](https://mlsanity.com/paper/2004.08515/) (2020)
- [Depth-aware CNN for RGB-D Segmentation](https://mlsanity.com/paper/1803.06791/) (2018)
- [Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06227/) (2020)
- [A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06811/) (2020)
- [Accurate RGB-D Salient Object Detection via Collaborative Learning](https://mlsanity.com/paper/2007.11782/) (2020)
- [RGB-D Salient Object Detection with Cross-Modality Modulation and Selection](https://mlsanity.com/paper/2007.07051/) (2020)
- [Displacement-Invariant Matching Cost Learning for Accurate Optical Flow Estimation](https://mlsanity.com/paper/2010.14851/) (2020)
- [Spatial Information Guided Convolution for Real-Time RGBD Semantic Segmentation](https://mlsanity.com/paper/2004.04534/) (2020)
- [Cross-Modal Weighting Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.04901/) (2020)
- [RGBD Salient Object Detection via Deep Fusion](https://mlsanity.com/paper/1607.03333/) (2016)
- [Progressively Guided Alternate Refinement Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2008.07064/) (2020)
- [Holopix50k: A Large-Scale In-the-wild Stereo Image Dataset](https://mlsanity.com/paper/2003.11172/) (2020)
- [Cascade Graph Neural Networks for RGB-D Salient Object Detection](https://mlsanity.com/paper/2008.03087/) (2020)
- [Bilateral Attention Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2004.14582/) (2020)
- [Adaptive Fusion for RGB-D Salient Object Detection](https://mlsanity.com/paper/1901.01369/) (2019)
- [Saliency Detection for Stereoscopic Images Based on Depth Confidence Analysis and Multiple Cues Fusion](https://mlsanity.com/paper/1710.05174/) (2017)
- [Is Depth Really Necessary for Salient Object Detection?](https://mlsanity.com/paper/2006.00269/) (2020)
- [Displacement-Invariant Cost Computation for Efficient Stereo Matching](https://mlsanity.com/paper/2012.00899/) (2020)

## Built on by (most cited first, 30 of 50)

- [Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications](https://mlsanity.com/paper/2304.05750/) (2023)
- [Implicit Motion Handling for Video Camouflaged Object Detection](https://mlsanity.com/paper/2203.07363/) (2022)
- [DFormer: Rethinking RGBD Representation Learning for Semantic Segmentation](https://mlsanity.com/paper/2309.09668/) (2023)
- [VSCode: General Visual Salient and Camouflaged Object Detection with 2D Prompt Learning](https://mlsanity.com/paper/2311.15011/) (2023)
- [Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction](https://mlsanity.com/paper/2112.13528/) (2021)
- [Source-free Depth for Object Pop-out](https://mlsanity.com/paper/2212.05370/) (2022)
- [CAVER: Cross-Modal View-Mixed Transformer for Bi-Modal Salient Object Detection](https://mlsanity.com/paper/2112.02363/) (2021)
- [HiDAnet: RGB-D Salient Object Detection via Hierarchical Depth Awareness](https://mlsanity.com/paper/2301.07405/) (2023)
- [VST++: Efficient and Stronger Visual Saliency Transformer](https://mlsanity.com/paper/2310.11725/) (2023)
- [Glass Segmentation with RGB-Thermal Image Pairs](https://mlsanity.com/paper/2204.05453/) (2022)
- [RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation](https://mlsanity.com/paper/2309.15082/) (2023)
- [Exploring Deeper! Segment Anything Model with Depth Perception for Camouflaged Object Detection](https://mlsanity.com/paper/2407.12339/) (2024)
- [Mutual Information-driven Triple Interaction Network for Efficient Image Dehazing](https://mlsanity.com/paper/2308.06998/) (2023)
- [Towards Diverse Binary Segmentation via A Simple yet General Gated Network](https://mlsanity.com/paper/2303.10396/) (2023)
- [Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flow](https://mlsanity.com/paper/2403.07432/) (2024)
- [Mutual-Guided Dynamic Network for Image Fusion](https://mlsanity.com/paper/2308.12538/) (2023)
- [Integrating Extra Modality Helps Segmentor Find Camouflaged Objects Well](https://mlsanity.com/paper/2502.14471/) (2025)
- [Improving SAM for Camouflaged Object Detection via Dual Stream Adapters](https://mlsanity.com/paper/2503.06042/) (2025)
- [Robust RGB-D Fusion for Saliency Detection](https://mlsanity.com/paper/2208.01762/) (2022)
- [Object Segmentation by Mining Cross-Modal Semantics](https://mlsanity.com/paper/2305.10469/) (2023)
- [CalibNet: Dual-branch Cross-modal Calibration for RGB-D Salient Instance Segmentation](https://mlsanity.com/paper/2307.08098/) (2023)
- [Enhancing Feature Diversity Boosts Channel-Adaptive Vision Transformers](https://mlsanity.com/paper/2405.16419/) (2024)
- [Mutual Information Regularization for Weakly-supervised RGB-D Salient Object Detection](https://mlsanity.com/paper/2306.03630/) (2023)
- [Inspiring the Next Generation of Segment Anything Models: Comprehensively Evaluate SAM and SAM 2 with Diverse Prompts Towards Context-Dependent Concepts under Different Scenes](https://mlsanity.com/paper/2412.01240/) (2024)
- [Transmission-Guided Bayesian Generative Model for Smoke Segmentation](https://mlsanity.com/paper/2303.00900/) (2023)
- [ViDSOD-100: A New Dataset and a Baseline Model for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2406.12536/) (2024)
- [Learning from Pixel-Level Noisy Label : A New Perspective for Light Field Saliency Detection](https://mlsanity.com/paper/2204.13456/) (2022)
- [Multimodal Informative ViT: Information Aggregation and Distribution for Hyperspectral and LiDAR Classification](https://mlsanity.com/paper/2401.03179/) (2024)
- [Rethinking Object Saliency Ranking: A Novel Whole-flow Processing Paradigm](https://mlsanity.com/paper/2312.03226/) (2023)
- [DADM: Dual Alignment of Domain and Modality for Face Anti-spoofing](https://mlsanity.com/paper/2503.00429/) (2025)

## Abstract (arXiv)

Existing RGB-D saliency detection models do not explicitly encourage RGB and depth to achieve effective multi-modal learning. In this paper, we introduce a novel multi-stage cascaded learning framework via mutual information minimization to "explicitly" model the multi-modal information between RGB image and depth data. Specifically, we first map the feature of each mode to a lower dimensional feature vector, and adopt mutual information minimization as a regularizer to reduce the redundancy between appearance features from RGB and geometric features from depth. We then perform multi-stage cascaded learning to impose the mutual information minimization constraint at every stage of the network. Extensive experiments on benchmark RGB-D saliency datasets illustrate the effectiveness of our framework. Further, to prosper the development of this field, we contribute the largest (7x larger than NJU2K) dataset, which contains 15,625 image pairs with high quality polygon-/scribble-/object-/instance-/rank-level annotations. Based on these rich labels, we additionally construct four new benchmarks with strong baselines and observe some interesting phenomena, which can motivate future model design. Source code and dataset are available at "https://github.com/JingZhang617/cascaded_rgbd_sod".