# A Unified RGB-T Saliency Detection Benchmark: Dataset, Baselines, Analysis and A Novel Approach

arXiv: https://arxiv.org/abs/1701.02829 · HTML: https://mlsanity.com/paper/1701.02829/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jan 2017.

## Built on by (most cited first, 30 of 51)

- [SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection](https://mlsanity.com/paper/2204.05585/) (2022)
- [VSCode: General Visual Salient and Camouflaged Object Detection with 2D Prompt Learning](https://mlsanity.com/paper/2311.15011/) (2023)
- [CAVER: Cross-Modal View-Mixed Transformer for Bi-Modal Salient Object Detection](https://mlsanity.com/paper/2112.02363/) (2021)
- [Does Thermal Really Always Matter for RGB-T Salient Object Detection?](https://mlsanity.com/paper/2210.04266/) (2022)
- [An Interactively Reinforced Paradigm for Joint Infrared-Visible Image Fusion and Saliency Object Detection](https://mlsanity.com/paper/2305.09999/) (2023)
- [VST++: Efficient and Stronger Visual Saliency Transformer](https://mlsanity.com/paper/2310.11725/) (2023)
- [HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection](https://mlsanity.com/paper/2301.03036/) (2023)
- [Glass Segmentation with RGB-Thermal Image Pairs](https://mlsanity.com/paper/2204.05453/) (2022)
- [Efficient Multimodal Semantic Segmentation via Dual-Prompt Learning](https://mlsanity.com/paper/2312.00360/) (2023)
- [Texture-guided Saliency Distilling for Unsupervised Salient Object Detection](https://mlsanity.com/paper/2207.05921/) (2022)
- [Alignment-Free RGBT Salient Object Detection: Semantics-guided Asymmetric Correlation Network and A Unified Benchmark](https://mlsanity.com/paper/2406.00917/) (2024)
- [Learning Adaptive Fusion Bank for Multi-modal Salient Object Detection](https://mlsanity.com/paper/2406.01127/) (2024)
- [UniRGB-IR: A Unified Framework for Visible-Infrared Semantic Tasks via Adapter Tuning](https://mlsanity.com/paper/2404.17360/) (2024)
- [TAFNet: A Three-Stream Adaptive Fusion Network for RGB-T Crowd Counting](https://mlsanity.com/paper/2202.08517/) (2022)
- [Visible-Thermal Multiple Object Tracking: Large-scale Video Dataset and Progressive Fusion Approach](https://mlsanity.com/paper/2408.00969/) (2024)
- [Position-Aware Relation Learning for RGB-Thermal Salient Object Detection](https://mlsanity.com/paper/2209.10158/) (2022)
- [Object Segmentation by Mining Cross-Modal Semantics](https://mlsanity.com/paper/2305.10469/) (2023)
- [Spectrum-driven Mixed-frequency Network for Hyperspectral Salient Object Detection](https://mlsanity.com/paper/2312.01060/) (2023)
- [Scribble-Supervised RGB-T Salient Object Detection](https://mlsanity.com/paper/2303.09733/) (2023)
- [M-SpecGene: Generalized Foundation Model for RGBT Multispectral Vision](https://mlsanity.com/paper/2507.16318/) (2025)
- [SSFam: Scribble Supervised Salient Object Detection Family](https://mlsanity.com/paper/2409.04817/) (2024)
- [Multispectral State-Space Feature Fusion: Bridging Shared and Cross-Parametric Interactions for Object Detection](https://mlsanity.com/paper/2507.14643/) (2025)
- [DifferSeg: Towards Diverse Multimodal Binary Segmentation via Differential Perception and Frequency Guidance](https://mlsanity.com/paper/2606.08906/) (2026)
- [Unified Unsupervised Salient Object Detection via Knowledge Transfer](https://mlsanity.com/paper/2404.14759/) (2024)
- [OmniSegmentor: A Flexible Multi-Modal Learning Framework for Semantic Segmentation](https://mlsanity.com/paper/2509.15096/) (2025)
- [Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging Scenarios](https://mlsanity.com/paper/2402.11826/) (2024)
- [KAN-SAM: Kolmogorov-Arnold Network Guided Segment Anything Model for RGB-T Salient Object Detection](https://mlsanity.com/paper/2504.05878/) (2025)
- [WXSOD: A Benchmark for Robust Salient Object Detection in Adverse Weather Conditions](https://mlsanity.com/paper/2508.12250/) (2025)
- [HVPNet: A Bio-Inspired Network for General Salient and Camouflaged Object Detection](https://mlsanity.com/paper/2606.31496/) (2026)
- [Mirror Complementary Transformer Network for RGB-thermal Salient Object Detection](https://mlsanity.com/paper/2207.03558/) (2022)

## Abstract (arXiv)

Despite significant progress, image saliency detection still remains a challenging task in complex scenes and environments. Integrating multiple different but complementary cues, like RGB and Thermal (RGB-T), may be an effective way for boosting saliency detection performance. The current research in this direction, however, is limited by the lack of a comprehensive benchmark. This work contributes such a RGB-T image dataset, which includes 821 spatially aligned RGB-T image pairs and their ground truth annotations for saliency detection purpose. The image pairs are with high diversity recorded under different scenes and environmental conditions, and we annotate 11 challenges on these image pairs for performing the challenge-sensitive analysis for different saliency detection algorithms. We also implement 3 kinds of baseline methods with different modality inputs to provide a comprehensive comparison platform. With this benchmark, we propose a novel approach, multi-task manifold ranking with cross-modality consistency, for RGB-T saliency detection. In particular, we introduce a weight for each modality to describe the reliability, and integrate them into the graph-based manifold ranking algorithm to achieve adaptive fusion of different source data. Moreover, we incorporate the cross-modality consistent constraints to integrate different modalities collaboratively. For the optimization, we design an efficient algorithm to iteratively solve several subproblems with closed-form solutions. Extensive experiments against other baseline methods on the newly created benchmark demonstrate the effectiveness of the proposed approach, and we also provide basic insights and potential future research directions for RGB-T saliency detection.