# SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection

arXiv: https://arxiv.org/abs/2204.05585 · HTML: https://mlsanity.com/paper/2204.05585/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 98 references that are in the map.
- First posted on arXiv 12 Apr 2022.

## Builds on (most cited first, 30 of 50)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks](https://mlsanity.com/paper/1907.06781/) (2019)
- [Salient Object Detection: A Benchmark](https://mlsanity.com/paper/1501.02741/) (2015)
- [UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders](https://mlsanity.com/paper/2004.05763/) (2020)
- [Towards High-Resolution Salient Object Detection](https://mlsanity.com/paper/1908.07274/) (2019)
- [CAT: Cross Attention in Vision Transformer](https://mlsanity.com/paper/2106.05786/) (2021)
- [JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection](https://mlsanity.com/paper/2004.08515/) (2020)
- [RGBT Salient Object Detection: A Large-scale Dataset and Benchmark](https://mlsanity.com/paper/2007.03262/) (2020)
- [A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06811/) (2020)
- [Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images](https://mlsanity.com/paper/2011.13144/) (2020)
- [Review of Visual Saliency Detection with Comprehensive Information](https://mlsanity.com/paper/1803.03391/) (2018)
- [RGB-T Image Saliency Detection via Collaborative Graph Learning](https://mlsanity.com/paper/1905.06741/) (2019)
- [Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion](https://mlsanity.com/paper/2103.11832/) (2021)
- [Accurate RGB-D Salient Object Detection via Collaborative Learning](https://mlsanity.com/paper/2007.11782/) (2020)
- [A Unified RGB-T Saliency Detection Benchmark: Dataset, Baselines, Analysis and A Novel Approach](https://mlsanity.com/paper/1701.02829/) (2017)
- [Nested Network with Two-Stream Pyramid for Salient Object Detection in Optical Remote Sensing Images](https://mlsanity.com/paper/1906.08462/) (2019)
- [RGB-D Salient Object Detection with Cross-Modality Modulation and Selection](https://mlsanity.com/paper/2007.07051/) (2020)

## Built on by (most cited first, 30 of 59)

- [VST++: Efficient and Stronger Visual Saliency Transformer](https://mlsanity.com/paper/2310.11725/) (2023)
- [HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection](https://mlsanity.com/paper/2301.03036/) (2023)
- [GraphMLP: A Graph MLP-Like Architecture for 3D Human Pose Estimation](https://mlsanity.com/paper/2206.06420/) (2022)
- [Alignment-Free RGBT Salient Object Detection: Semantics-guided Asymmetric Correlation Network and A Unified Benchmark](https://mlsanity.com/paper/2406.00917/) (2024)
- [Learning Adaptive Fusion Bank for Multi-modal Salient Object Detection](https://mlsanity.com/paper/2406.01127/) (2024)
- [Salient Object Detection in Optical Remote Sensing Images Driven by Transformer](https://mlsanity.com/paper/2309.08206/) (2023)
- [Position-Aware Relation Learning for RGB-Thermal Salient Object Detection](https://mlsanity.com/paper/2209.10158/) (2022)
- [Object Segmentation by Mining Cross-Modal Semantics](https://mlsanity.com/paper/2305.10469/) (2023)
- [Decomposed Guided Dynamic Filters for Efficient RGB-Guided Depth Completion](https://mlsanity.com/paper/2309.02043/) (2023)
- [Panoramic Video Salient Object Detection with Ambisonic Audio Guidance](https://mlsanity.com/paper/2211.14419/) (2022)
- [Mutual Information Regularization for Weakly-supervised RGB-D Salient Object Detection](https://mlsanity.com/paper/2306.03630/) (2023)
- [Unified-modal Salient Object Detection via Adaptive Prompt Learning](https://mlsanity.com/paper/2311.16835/) (2023)
- [Spectrum-driven Mixed-frequency Network for Hyperspectral Salient Object Detection](https://mlsanity.com/paper/2312.01060/) (2023)
- [Scribble-Supervised RGB-T Salient Object Detection](https://mlsanity.com/paper/2303.09733/) (2023)
- [Quality-aware Selective Fusion Network for V-D-T Salient Object Detection](https://mlsanity.com/paper/2405.07655/) (2024)
- [METER: a mobile vision transformer architecture for monocular depth estimation](https://mlsanity.com/paper/2403.08368/) (2024)
- [Hybrid-Fusion Transformer for Multisequence MRI](https://mlsanity.com/paper/2311.01308/) (2023)
- [SSFam: Scribble Supervised Salient Object Detection Family](https://mlsanity.com/paper/2409.04817/) (2024)
- [Part-Whole Relational Fusion Towards Multi-Modal Scene Understanding](https://mlsanity.com/paper/2410.14944/) (2024)
- [MambaSOD: Dual Mamba-Driven Cross-Modal Fusion Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2410.15015/) (2024)
- [Lightweight RGB-D Salient Object Detection from a Speed-Accuracy Tradeoff Perspective](https://mlsanity.com/paper/2505.04758/) (2025)
- [Transformer-based RGB-T Tracking with Channel and Spatial Feature Fusion](https://mlsanity.com/paper/2405.03177/) (2024)
- [SeaDATE: Remedy Dual-Attention Transformer with Semantic Alignment via Contrast Learning for Multimodal Object Detection](https://mlsanity.com/paper/2410.11358/) (2024)
- [KAN-SAM: Kolmogorov-Arnold Network Guided Segment Anything Model for RGB-T Salient Object Detection](https://mlsanity.com/paper/2504.05878/) (2025)
- [Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2206.03105/) (2022)
- [Mirror Complementary Transformer Network for RGB-thermal Salient Object Detection](https://mlsanity.com/paper/2207.03558/) (2022)
- [ComPtr: Towards Diverse Bi-source Dense Prediction Tasks via A Simple yet General Complementary Transformer](https://mlsanity.com/paper/2307.12349/) (2023)
- [Benchmarking Ultra-High-Definition Image Reflection Removal](https://mlsanity.com/paper/2308.00265/) (2023)
- [All in One: RGB, RGB-D, and RGB-T Salient Object Detection](https://mlsanity.com/paper/2311.14746/) (2023)
- [FDiff-Fusion:Denoising diffusion fusion network based on fuzzy learning for 3D medical image segmentation](https://mlsanity.com/paper/2408.02075/) (2024)

## Abstract (arXiv)

Convolutional neural networks (CNNs) are good at extracting contexture features within certain receptive fields, while transformers can model the global long-range dependency features. By absorbing the advantage of transformer and the merit of CNN, Swin Transformer shows strong feature representation ability. Based on it, we propose a cross-modality fusion model SwinNet for RGB-D and RGB-T salient object detection. It is driven by Swin Transformer to extract the hierarchical features, boosted by attention mechanism to bridge the gap between two modalities, and guided by edge information to sharp the contour of salient object. To be specific, two-stream Swin Transformer encoder first extracts multi-modality features, and then spatial alignment and channel re-calibration module is presented to optimize intra-level cross-modality features. To clarify the fuzzy boundary, edge-guided decoder achieves inter-level cross-modality fusion under the guidance of edge features. The proposed model outperforms the state-of-the-art models on RGB-D and RGB-T datasets, showing that it provides more insight into the cross-modality complementarity task.