mlsanityData to 14 Oct 2026

Depth Quality-Inspired Feature Manipulation for Efficient RGB-D and Video Salient Object Detection

arXiv 2208.03918 ↗First posted 8 Aug 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recently CNN-based RGB-D salient object detection (SOD) has obtained significant improvement on detection accuracy. However, existing models often fail to perform well in terms of efficiency and accuracy simultaneously. This hinders their potential applications on mobile devices as well as many real-world problems. To bridge the accuracy gap between lightweight and large models for RGB-D SOD, in this paper, an efficient module that can greatly improve the accuracy but adds little computation is proposed. Inspired by the fact that depth quality is a key factor influencing the accuracy, we propose an efficient depth quality-inspired feature manipulation (DQFM) process, which can dynamically filter depth features according to depth quality. The proposed DQFM resorts to the alignment of low-level RGB and depth features, as well as holistic attention of the depth stream to explicitly control and enhance cross-modal fusion. We embed DQFM to obtain an efficient lightweight RGB-D SOD model called DFM-Net, where we in addition design a tailored depth backbone and a two-stage decoder as basic parts. Extensive experimental results on nine RGB-D datasets demonstrate that our DFM-Net outperforms recent efficient models, running at about 20 FPS on CPU with only 8.5Mb model size, and meanwhile being 2.9/2.4 times faster and 6.7/3.1 times smaller than the latest best models A2dele and MobileSal. It also maintains state-of-the-art accuracy when even compared to non-efficient models. Interestingly, further statistics and analyses verify the ability of DQFM in distinguishing depth maps of various qualities without any quality labels. Last but not least, we further apply DFM-Net to deal with video SOD (VSOD), achieving comparable performance against recent efficient models while being 3/2.3 times faster/smaller than the prior best in this field. Our code is available at https://github.com/zwbx/DFM-Net.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 64 of its 110 references that are in the map. First posted on arXiv 8 Aug 2022.

Builds on

64 of 110 references.

References · 64

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2017Squeeze-and-Excitation Networkscited by 4,789
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2016Pyramid Scene Parsing Networkcited by 2,479
  10. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  11. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  12. 2017Structure-measure: A New Way to Evaluate Foreground Mapscited by 480
38 more
  1. 2020PraNet: Parallel Reverse Attention Network for Polyp Segmentationcited by 460
  2. 2018Enhanced-alignment Measure for Binary Foreground Map Evaluationcited by 370
  3. 2021Visual Saliency Transformercited by 175
  4. 2019Salient Object Detection in the Deep Learning Era: An In-Depth Surveycited by 164
  5. 2019Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarkscited by 133
  6. 2015Salient Object Detection: A Benchmarkcited by 131
  7. 2016The Importance of Skip Connections in Biomedical Image Segmentationcited by 126
  8. 2019Fast-SCNN: Fast Semantic Segmentation Networkcited by 125
  9. 2020UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoderscited by 98
  10. 2017Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detectioncited by 93
  11. 2014Salient Object Detection: A Discriminative Regional Feature Integration Approachcited by 89
  12. 2022Video Polyp Segmentation: A Deep Learning Perspectivecited by 82
  13. 2020JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detectioncited by 79
  14. 2021RobustART: Benchmarking Robustness on Architecture Design and Training Techniquescited by 79
  15. 2021Full-Duplex Strategy for Video Object Segmentationcited by 78
  16. 2021Progressively Normalized Self-Attention Network for Video Polyp Segmentationcited by 75
  17. 2020RGB-D Salient Object Detection: A Surveycited by 73
  18. 2019Bi-Directional Cascade Network for Perceptual Edge Detectioncited by 72
  19. 2020Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detectioncited by 72
  20. 2021Specificity-preserving RGB-D Saliency Detectioncited by 70
  21. 2020A Single Stream Network for Robust and Real-time RGB-D Salient Object Detectioncited by 67
  22. 2020Siamese Network for RGB-D Salient Object Detection and Beyondcited by 67
  23. 2018Review of Visual Saliency Detection with Comprehensive Informationcited by 60
  24. 2020Uncertainty Inspired RGB-D Saliency Detectioncited by 56
  25. 2021Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusioncited by 56
  26. 2019Semi-Supervised Video Salient Object Detection Using Pseudo-Labelscited by 54
  27. 2020Re-thinking Co-Salient Object Detectioncited by 54
  28. 2020Accurate RGB-D Salient Object Detection via Collaborative Learningcited by 52
  29. 2020RGB-D Salient Object Detection with Cross-Modality Modulation and Selectioncited by 50
  30. 2021RGB-D Saliency Detection via Cascaded Mutual Information Minimizationcited by 50
  31. 2020Cross-Modal Weighting Network for RGB-D Salient Object Detectioncited by 47
  32. 2016RGBD Salient Object Detection via Deep Fusioncited by 46
  33. 2017Video Salient Object Detection via Fully Convolutional Networkscited by 46
  34. 2020DPANet: Depth Potentiality-Aware Gated Attention Network for RGB-D Salient Object Detectioncited by 45
  35. 2021TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Networkcited by 45
  36. 2020Progressively Guided Alternate Refinement Network for RGB-D Salient Object Detectioncited by 40
  37. 2020Learning Selective Mutual Attention and Contrast for RGB-D Saliency Detectioncited by 38
  38. 2020Cascade Graph Neural Networks for RGB-D Salient Object Detectioncited by 36

50 most cited of 64 All 64 (JSON)

Built on by

Not cited yet.