# Salient Object Detection From Arbitrary Modalities

arXiv: https://arxiv.org/abs/2405.03352 · HTML: https://mlsanity.com/paper/2405.03352/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 75 references that are in the map.
- First posted on arXiv 6 May 2024.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks](https://mlsanity.com/paper/1907.06781/) (2019)
- [Pyramid Feature Attention Network for Saliency detection](https://mlsanity.com/paper/1903.00179/) (2019)
- [RGBT Salient Object Detection: A Large-scale Dataset and Benchmark](https://mlsanity.com/paper/2007.03262/) (2020)
- [SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection](https://mlsanity.com/paper/2204.05585/) (2022)
- [RGB-T Image Saliency Detection via Collaborative Graph Learning](https://mlsanity.com/paper/1905.06741/) (2019)
- [VSCode: General Visual Salient and Camouflaged Object Detection with 2D Prompt Learning](https://mlsanity.com/paper/2311.15011/) (2023)
- [A Unified RGB-T Saliency Detection Benchmark: Dataset, Baselines, Analysis and A Novel Approach](https://mlsanity.com/paper/1701.02829/) (2017)
- [Cross-Modal Weighting Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.04901/) (2020)
- [CAVER: Cross-Modal View-Mixed Transformer for Bi-Modal Salient Object Detection](https://mlsanity.com/paper/2112.02363/) (2021)
- [CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detection](https://mlsanity.com/paper/2210.02843/) (2022)
- [Does Thermal Really Always Matter for RGB-T Salient Object Detection?](https://mlsanity.com/paper/2210.04266/) (2022)
- [Frequency Perception Network for Camouflaged Object Detection](https://mlsanity.com/paper/2308.08924/) (2023)
- [RGB-D Salient Object Detection via 3D Convolutional Neural Networks](https://mlsanity.com/paper/2101.10241/) (2021)
- [Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network](https://mlsanity.com/paper/1502.06796/) (2015)
- [HiDAnet: RGB-D Salient Object Detection via Hierarchical Depth Awareness](https://mlsanity.com/paper/2301.07405/) (2023)
- [Adaptive Fusion for RGB-D Salient Object Detection](https://mlsanity.com/paper/1901.01369/) (2019)
- [MobileSal: Extremely Efficient RGB-D Salient Object Detection](https://mlsanity.com/paper/2012.13095/) (2020)
- [SAC-Net: Spatial Attenuation Context for Salient Object Detection](https://mlsanity.com/paper/1903.10152/) (2019)
- [BTS-Net: Bi-directional Transfer-and-Selection Network For RGB-D Salient Object Detection](https://mlsanity.com/paper/2104.01784/) (2021)
- [Progressive Self-Guided Loss for Salient Object Detection](https://mlsanity.com/paper/2101.02412/) (2021)
- [Data-Level Recombination and Lightweight Fusion Scheme for RGB-D Salient Object Detection](https://mlsanity.com/paper/2009.05102/) (2020)
- [Middle-level Fusion for Lightweight RGB-D Salient Object Detection](https://mlsanity.com/paper/2104.11543/) (2021)
- [GLID: Pre-training a Generalist Encoder-Decoder Vision Model](https://mlsanity.com/paper/2404.07603/) (2024)

## Built on by (most cited first, 2 of 2)

- [SSFam: Scribble Supervised Salient Object Detection Family](https://mlsanity.com/paper/2409.04817/) (2024)
- [Salient Object Detection in Complex Weather Conditions via Noise Indicators](https://mlsanity.com/paper/2512.10592/) (2025)

## Abstract (arXiv)

Toward desirable saliency prediction, the types and numbers of inputs for a salient object detection (SOD) algorithm may dynamically change in many real-life applications. However, existing SOD algorithms are mainly designed or trained for one particular type of inputs, failing to be generalized to other types of inputs. Consequentially, more types of SOD algorithms need to be prepared in advance for handling different types of inputs, raising huge hardware and research costs. Differently, in this paper, we propose a new type of SOD task, termed Arbitrary Modality SOD (AM SOD). The most prominent characteristics of AM SOD are that the modality types and modality numbers will be arbitrary or dynamically changed. The former means that the inputs to the AM SOD algorithm may be arbitrary modalities such as RGB, depths, or even any combination of them. While, the latter indicates that the inputs may have arbitrary modality numbers as the input type is changed, e.g. single-modality RGB image, dual-modality RGB-Depth (RGB-D) images or triple-modality RGB-Depth-Thermal (RGB-D-T) images. Accordingly, a preliminary solution to the above challenges, i.e. a modality switch network (MSN), is proposed in this paper. In particular, a modality switch feature extractor (MSFE) is first designed to extract discriminative features from each modality effectively by introducing some modality indicators, which will generate some weights for modality switching. Subsequently, a dynamic fusion module (DFM) is proposed to adaptively fuse features from a variable number of modalities based on a novel Transformer structure. Finally, a new dataset, named AM-XD, is constructed to facilitate research on AM SOD. Extensive experiments demonstrate that our AM SOD method can effectively cope with changes in the type and number of input modalities for robust salient object detection.