mlsanityData to 14 Oct 2026

Mutual Information Regularization for Weakly-supervised RGB-D Salient Object Detection

arXiv 2306.03630 ↗First posted 6 Jun 2023cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a weakly-supervised RGB-D salient object detection model via scribble supervision. Specifically, as a multimodal learning task, we focus on effective multimodal representation learning via inter-modal mutual information regularization. In particular, following the principle of disentangled representation learning, we introduce a mutual information upper bound with a mutual information minimization regularizer to encourage the disentangled representation of each modality for salient object detection. Based on our multimodal representation learning framework, we introduce an asymmetric feature extractor for our multimodal data, which is proven more effective than the conventional symmetric backbone setting. We also introduce multimodal variational auto-encoder as stochastic prediction refinement techniques, which takes pseudo labels from the first training stage as supervision and generates refined prediction. Experimental results on benchmark RGB-D salient object detection datasets verify both effectiveness of our explicit multimodal disentangled representation learning method and the stochastic prediction refinement strategy, achieving comparable performance with the state-of-the-art fully supervised models. Our code and data are available at: https://github.com/baneitixiaomai/MIRV.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 64 of its 137 references that are in the map. First posted on arXiv 6 Jun 2023.

Builds on

64 of 137 references.

References · 64

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  6. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  7. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  8. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  9. 2019On Variational Bounds of Mutual Informationcited by 559
  10. 2017Structure-measure: A New Way to Evaluate Foreground Mapscited by 480
  11. 2018Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentationcited by 321
  12. 2019Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relationscited by 247
38 more
  1. 2020CLUB: A Contrastive Log-ratio Upper Bound of Mutual Informationcited by 235
  2. 2019Cascaded Partial Decoder for Fast and Accurate Salient Object Detectioncited by 216
  3. 2018Multimodal Generative Models for Scalable Weakly-Supervised Learningcited by 200
  4. 2019F3Net: Fusion, Feedback and Focus for Salient Object Detectioncited by 176
  5. 2021Visual Saliency Transformercited by 175
  6. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  7. 2015Adding Gradient Noise Improves Learning for Very Deep Networkscited by 145
  8. 2021Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentationcited by 134
  9. 2019Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarkscited by 133
  10. 2018Normalized Cut Loss for Weakly-supervised CNN Segmentationcited by 112
  11. 2020UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoderscited by 98
  12. 2020Weakly-Supervised Salient Object Detection via Scribble Annotationscited by 96
  13. 2020JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detectioncited by 79
  14. 2021Specificity-preserving RGB-D Saliency Detectioncited by 70
  15. 2014Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictionscited by 67
  16. 2020A Single Stream Network for Robust and Real-time RGB-D Salient Object Detectioncited by 67
  17. 2021Reducing Information Bottleneck for Weakly Supervised Semantic Segmentationcited by 65
  18. 2022SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detectioncited by 59
  19. 2021Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Predictioncited by 57
  20. 2021Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusioncited by 56
  21. 2020Structure-Consistent Weakly Supervised Salient Object Detection with Local Saliency Coherencecited by 55
  22. 2019Learning Disentangled Representations via Mutual Information Estimationcited by 52
  23. 2018Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspectivecited by 50
  24. 2021RGB-D Saliency Detection via Cascaded Mutual Information Minimizationcited by 50
  25. 2021Pseudo-mask Matters in Weakly-supervised Semantic Segmentationcited by 48
  26. 2020Cross-Modal Weighting Network for RGB-D Salient Object Detectioncited by 47
  27. 2022Tree Energy Loss: Towards Sparsely Annotated Semantic Segmentationcited by 47
  28. 2013Equitability, mutual information, and the maximal information coefficientcited by 45
  29. 2022CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detectioncited by 38
  30. 2021A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representationscited by 37
  31. 2021RGB-D Salient Object Detection via 3D Convolutional Neural Networkscited by 36
  32. 2020Adaptive Graph Convolutional Network with Attention Graph Clustering for Co-saliency Detectioncited by 35
  33. 2018Weakly Supervised Salient Object Detection Using Image Labelscited by 32
  34. 2021All you need are a few pixels: semantic segmentation with PixelPickcited by 31
  35. 2019Adaptive Fusion for RGB-D Salient Object Detectioncited by 29
  36. 2020Learning Noise-Aware Encoder-Decoder from Noisy Labels by Alternating Back-Propagation for Saliency Detectioncited by 29
  37. 2021Weakly Supervised Video Salient Object Detectioncited by 28
  38. 2021MFNet: Multi-filter Directive Network for Weakly Supervised Salient Object Detectioncited by 27

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 1; 2026: 1.

9 papers.

  1. 2023Contrastive Conditional Latent Diffusion for Audio-visual Segmentationcited by 31
  2. 2024Learning Adaptive Fusion Bank for Multi-modal Salient Object Detectioncited by 19
  3. 2024SSFam: Scribble Supervised Salient Object Detection Familycited by 6
  4. 2023M3Net: Multilevel, Mixed and Multistage Attention Network for Salient Object Detectioncited by 4
  5. 2024Constructing Enhanced Mutual Information for Online Class-Incremental Learningcited by 2
  6. 2024A Volumetric Saliency Guided Image Summarization for RGB-D Indoor Scene Classificationcited by 1
  7. 2024FIPGNet:Pyramid grafting network with feature interaction strategiescited by 0
  8. 2025Dual Mutual Learning Network with Global-local Awareness for RGB-D Salient Object Detectioncited by 0
  9. 2026Weakly-Supervised RGB-D Salient Object Detection via SAM-driven Pseudo Annotation and State Space Interaction-based Diffusioncited by 0