# DFTR: Depth-supervised Fusion Transformer for Salient Object Detection

arXiv: https://arxiv.org/abs/2203.06429 · HTML: https://mlsanity.com/paper/2203.06429/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 74 references that are in the map.
- First posted on arXiv 12 Mar 2022.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [Multi-scale Interactive Network for Salient Object Detection](https://mlsanity.com/paper/2007.09062/) (2020)
- [The Secrets of Salient Object Segmentation](https://mlsanity.com/paper/1406.2807/) (2014)
- [Visual Saliency Based on Multiscale Deep Features](https://mlsanity.com/paper/1503.08663/) (2015)
- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks](https://mlsanity.com/paper/1907.06781/) (2019)
- [Pyramid Feature Attention Network for Saliency detection](https://mlsanity.com/paper/1903.00179/) (2019)
- [Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2105.08965/) (2021)
- [Suppress and Balance: A Simple Gated Network for Salient Object Detection](https://mlsanity.com/paper/2007.08074/) (2020)
- [Label Decoupling Framework for Salient Object Detection](https://mlsanity.com/paper/2008.11048/) (2020)
- [Improving 3D Object Detection with Channel-wise Transformer](https://mlsanity.com/paper/2108.10723/) (2021)
- [Highly Efficient Salient Object Detection with 100K Parameters](https://mlsanity.com/paper/2003.05643/) (2020)
- [A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06811/) (2020)
- [Siamese Network for RGB-D Salient Object Detection and Beyond](https://mlsanity.com/paper/2008.12134/) (2020)

## Built on by (most cited first, 5 of 5)

- [Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Review](https://mlsanity.com/paper/2303.06471/) (2023)
- [Position-Aware Relation Learning for RGB-Thermal Salient Object Detection](https://mlsanity.com/paper/2209.10158/) (2022)
- [Spectrum-driven Mixed-frequency Network for Hyperspectral Salient Object Detection](https://mlsanity.com/paper/2312.01060/) (2023)
- [Mirror Complementary Transformer Network for RGB-thermal Salient Object Detection](https://mlsanity.com/paper/2207.03558/) (2022)
- [A positive feedback method based on F-measure value for Salient Object Detection](https://mlsanity.com/paper/2304.14619/) (2023)

## Abstract (arXiv)

Automated salient object detection (SOD) plays an increasingly crucial role in many computer vision applications. By reformulating the depth information as supervision rather than as input, depth-supervised convolutional neural networks (CNN) have achieved promising results on both RGB and RGB-D SOD scenarios with the merits of no requirements for extra depth networks and depth inputs in the inference stage. This paper, for the first time, seeks to expand the applicability of depth supervision to the Transformer architecture. Specifically, we develop a Depth-supervised Fusion TRansformer (DFTR), to further improve the accuracy of both RGB and RGB-D SOD. The proposed DFTR involves three primary features: 1) DFTR, to the best of our knowledge, is the first pure Transformer-based model for depth-supervised SOD; 2) A multi-scale feature aggregation (MFA) module is proposed to fully exploit the multi-scale features encoded by the Swin Transformer in a coarse-to-fine manner; 3) To enable bidirectional information flow across different streams of features, a novel multi-stage feature fusion (MFF) module is further integrated into our DFTR with the emphasis on salient regions at different network learning stages. We extensively evaluate the proposed DFTR on ten benchmarking datasets. Experimental results show that our DFTR consistently outperforms the existing state-of-the-art methods for both RGB and RGB-D SOD tasks. The code and model will be made publicly available.