# All in One: RGB, RGB-D, and RGB-T Salient Object Detection

arXiv: https://arxiv.org/abs/2311.14746 · HTML: https://mlsanity.com/paper/2311.14746/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 23 Nov 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [PVT v2: Improved Baselines with Pyramid Vision Transformer](https://mlsanity.com/paper/2106.13797/) (2021)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks](https://mlsanity.com/paper/1907.06781/) (2019)
- [Salient Object Detection via Integrity Learning](https://mlsanity.com/paper/2101.07663/) (2021)
- [Salient Object Detection: A Benchmark](https://mlsanity.com/paper/1501.02741/) (2015)
- [RGBT Salient Object Detection: A Large-scale Dataset and Benchmark](https://mlsanity.com/paper/2007.03262/) (2020)
- [Siamese Network for RGB-D Salient Object Detection and Beyond](https://mlsanity.com/paper/2008.12134/) (2020)
- [SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection](https://mlsanity.com/paper/2204.05585/) (2022)
- [RGB-T Image Saliency Detection via Collaborative Graph Learning](https://mlsanity.com/paper/1905.06741/) (2019)
- [A Unified RGB-T Saliency Detection Benchmark: Dataset, Baselines, Analysis and A Novel Approach](https://mlsanity.com/paper/1701.02829/) (2017)
- [CAVER: Cross-Modal View-Mixed Transformer for Bi-Modal Salient Object Detection](https://mlsanity.com/paper/2112.02363/) (2021)
- [Does Thermal Really Always Matter for RGB-T Salient Object Detection?](https://mlsanity.com/paper/2210.04266/) (2022)
- [Depth Quality-Inspired Feature Manipulation for Efficient RGB-D Salient Object Detection](https://mlsanity.com/paper/2107.01779/) (2021)
- [Recursive Contour Saliency Blending Network for Accurate Salient Object Detection](https://mlsanity.com/paper/2105.13865/) (2021)
- [SiaTrans: Siamese Transformer Network for RGB-D Salient Object Detection with Depth Image Classification](https://mlsanity.com/paper/2207.04224/) (2022)

## Built on by (most cited first, 3 of 3)

- [SSFam: Scribble Supervised Salient Object Detection Family](https://mlsanity.com/paper/2409.04817/) (2024)
- [Salient Object Detection From Arbitrary Modalities](https://mlsanity.com/paper/2405.03352/) (2024)
- [Modality Prompts for Arbitrary Modality Salient Object Detection](https://mlsanity.com/paper/2405.03351/) (2024)

## Abstract (arXiv)

Salient object detection (SOD) aims to identify the most attractive objects within an image. Depending on the type of data being detected, SOD can be categorized into various forms, including RGB, RGB-D (Depth), RGB-T (Thermal) and light field SOD. Previous researches have focused on saliency detection with individual data type. If the RGB-D SOD model is forced to detect RGB-T data it will perform poorly. We propose an innovative model framework that provides a unified solution for the salient object detection task of three types of data (RGB, RGB-D, and RGB-T). The three types of data can be handled in one model (all in one) with the same weight parameters. In this framework, the three types of data are concatenated in an ordered manner within a single input batch, and features are extracted using a transformer network. Based on this framework, we propose an efficient lightweight SOD model, namely AiOSOD, which can detect any RGB, RGB-D, and RGB-T data with high speed (780FPS for RGB data, 485FPS for RGB-D or RGB-T data). Notably, with only 6.25M parameters, AiOSOD achieves excellent performance on RGB, RGB-D, and RGB-T datasets.