# HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection

arXiv: https://arxiv.org/abs/2301.03036 · HTML: https://mlsanity.com/paper/2301.03036/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 68 of its 135 references that are in the map.
- First posted on arXiv 8 Jan 2023.

## Builds on (most cited first, 30 of 68)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows](https://mlsanity.com/paper/2107.00652/) (2021)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [Efficient Attention: Attention with Linear Complexities](https://mlsanity.com/paper/1812.01243/) (2018)
- [Coordinate Attention for Efficient Mobile Network Design](https://mlsanity.com/paper/2103.02907/) (2021)
- [High-Resolution Representations for Labeling Pixels and Regions](https://mlsanity.com/paper/1904.04514/) (2019)
- [F3Net: Fusion, Feedback and Focus for Salient Object Detection](https://mlsanity.com/paper/1911.11445/) (2019)
- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [HRFormer: High-Resolution Transformer for Dense Prediction](https://mlsanity.com/paper/2110.09408/) (2021)
- [Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks](https://mlsanity.com/paper/1907.06781/) (2019)
- [Salient Object Detection: A Benchmark](https://mlsanity.com/paper/1501.02741/) (2015)
- [UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders](https://mlsanity.com/paper/2004.05763/) (2020)
- [Multi-Scale High-Resolution Vision Transformer for Semantic Segmentation](https://mlsanity.com/paper/2111.01236/) (2021)
- [Lite-HRNet: A Lightweight High-Resolution Network](https://mlsanity.com/paper/2104.06403/) (2021)
- [Learning Dynamic Routing for Semantic Segmentation](https://mlsanity.com/paper/2003.10401/) (2020)
- [RGB-D Salient Object Detection: A Survey](https://mlsanity.com/paper/2008.00230/) (2020)
- [Specificity-preserving RGB-D Saliency Detection](https://mlsanity.com/paper/2108.08162/) (2021)
- [RGBT Salient Object Detection: A Large-scale Dataset and Benchmark](https://mlsanity.com/paper/2007.03262/) (2020)
- [Siamese Network for RGB-D Salient Object Detection and Beyond](https://mlsanity.com/paper/2008.12134/) (2020)
- [Review of Visual Saliency Detection with Comprehensive Information](https://mlsanity.com/paper/1803.03391/) (2018)
- [SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection](https://mlsanity.com/paper/2204.05585/) (2022)

## Built on by (most cited first, 27 of 27)

- [VST++: Efficient and Stronger Visual Saliency Transformer](https://mlsanity.com/paper/2310.11725/) (2023)
- [Alignment-Free RGBT Salient Object Detection: Semantics-guided Asymmetric Correlation Network and A Unified Benchmark](https://mlsanity.com/paper/2406.00917/) (2024)
- [Learning Adaptive Fusion Bank for Multi-modal Salient Object Detection](https://mlsanity.com/paper/2406.01127/) (2024)
- [Object Segmentation by Mining Cross-Modal Semantics](https://mlsanity.com/paper/2305.10469/) (2023)
- [Unified-modal Salient Object Detection via Adaptive Prompt Learning](https://mlsanity.com/paper/2311.16835/) (2023)
- [Salient Object Detection in RGB-D Videos](https://mlsanity.com/paper/2310.15482/) (2023)
- [Quality-aware Selective Fusion Network for V-D-T Salient Object Detection](https://mlsanity.com/paper/2405.07655/) (2024)
- [SSFam: Scribble Supervised Salient Object Detection Family](https://mlsanity.com/paper/2409.04817/) (2024)
- [Part-Whole Relational Fusion Towards Multi-Modal Scene Understanding](https://mlsanity.com/paper/2410.14944/) (2024)
- [M3Net: Multilevel, Mixed and Multistage Attention Network for Salient Object Detection](https://mlsanity.com/paper/2309.08365/) (2023)
- [ComPtr: Towards Diverse Bi-source Dense Prediction Tasks via A Simple yet General Complementary Transformer](https://mlsanity.com/paper/2307.12349/) (2023)
- [SAMSOD: Rethinking SAM Optimization for RGB-T Salient Object Detection](https://mlsanity.com/paper/2510.03689/) (2025)
- [Deep Fourier-embedded Network for RGB and Thermal Salient Object Detection](https://mlsanity.com/paper/2411.18409/) (2024)
- [STENet: Superpixel Token Enhancing Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2603.21999/) (2026)
- [RA-SOD: Reliability-Aware RGB-T Salient Object Detection under Modality Degradation](https://mlsanity.com/paper/2609.12622/) (2026)
- [A Volumetric Saliency Guided Image Summarization for RGB-D Indoor Scene Classification](https://mlsanity.com/paper/2401.16227/) (2024)
- [Leveraging High-Resolution Features for Improved Deep Hashing-based Image Retrieval](https://mlsanity.com/paper/2403.13747/) (2024)
- [Intra-task Mutual Attention based Vision Transformer for Few-Shot Learning](https://mlsanity.com/paper/2405.03109/) (2024)
- [Class-relevant Patch Embedding Selection for Few-Shot Image Classification](https://mlsanity.com/paper/2405.03722/) (2024)
- [Efficient Fourier Filtering Network with Contrastive Learning for AAV-based Unaligned Bimodal Salient Object Detection](https://mlsanity.com/paper/2411.03728/) (2024)
- [FIPGNet:Pyramid grafting network with feature interaction strategies](https://mlsanity.com/paper/2407.04085/) (2024)
- [SSNet: Saliency Prior and State Space Model-based Network for Salient Object Detection in RGB-D Images](https://mlsanity.com/paper/2503.02270/) (2025)
- [Unleashing the Power of Motion and Depth: A Selective Fusion Strategy for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2507.21857/) (2025)
- [RSONet: Region-guided Selective Optimization Network for RGB-T Salient Object Detection](https://mlsanity.com/paper/2603.12685/) (2026)
- [Modality-Agnostic Prompt Learning for Multi-Modal Camouflaged Object Detection](https://mlsanity.com/paper/2604.12380/) (2026)
- [M4-SAM: Multi-Modal Mixture-of-Experts with Memory-Augmented SAM for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2605.11760/) (2026)
- [S2A:Semantic-to-Spatial Alignment for Alignment-Free RGB-T Salient Object Detection](https://mlsanity.com/paper/2609.27413/) (2026)

## Abstract (arXiv)

The High-Resolution Transformer (HRFormer) can maintain high-resolution representation and share global receptive fields. It is friendly towards salient object detection (SOD) in which the input and output have the same resolution. However, two critical problems need to be solved for two-modality SOD. One problem is two-modality fusion. The other problem is the HRFormer output's fusion. To address the first problem, a supplementary modality is injected into the primary modality by using global optimization and an attention mechanism to select and purify the modality at the input level. To solve the second problem, a dual-direction short connection fusion module is used to optimize the output features of HRFormer, thereby enhancing the detailed representation of objects at the output level. The proposed model, named HRTransNet, first introduces an auxiliary stream for feature extraction of supplementary modality. Then, features are injected into the primary modality at the beginning of each multi-resolution branch. Next, HRFormer is applied to achieve forwarding propagation. Finally, all the output features with different resolutions are aggregated by intra-feature and inter-feature interactive transformers. Application of the proposed model results in impressive improvement for driving two-modality SOD tasks, e.g., RGB-D, RGB-T, and light field SOD.https://github.com/liuzywen/HRTransNet