# CAVER: Cross-Modal View-Mixed Transformer for Bi-Modal Salient Object Detection

arXiv: https://arxiv.org/abs/2112.02363 · HTML: https://mlsanity.com/paper/2112.02363/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 98 references that are in the map.
- First posted on arXiv 4 Dec 2021.

## Builds on (most cited first, 30 of 58)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Bag of Tricks for Image Classification with Convolutional Neural Networks](https://mlsanity.com/paper/1812.01187/) (2018)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [Multi-scale Interactive Network for Salient Object Detection](https://mlsanity.com/paper/2007.09062/) (2020)
- [Concealed Object Detection](https://mlsanity.com/paper/2102.10274/) (2021)
- [F3Net: Fusion, Feedback and Focus for Salient Object Detection](https://mlsanity.com/paper/1911.11445/) (2019)
- [Salient Object Detection in the Deep Learning Era: An In-Depth Survey](https://mlsanity.com/paper/1904.09146/) (2019)
- [Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks](https://mlsanity.com/paper/1907.06781/) (2019)
- [Suppress and Balance: A Simple Gated Network for Salient Object Detection](https://mlsanity.com/paper/2007.08074/) (2020)
- [Label Decoupling Framework for Salient Object Detection](https://mlsanity.com/paper/2008.11048/) (2020)

## Built on by (most cited first, 30 of 47)

- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [VSCode: General Visual Salient and Camouflaged Object Detection with 2D Prompt Learning](https://mlsanity.com/paper/2311.15011/) (2023)
- [Learning Modality-agnostic Representation for Semantic Segmentation from Any Modalities](https://mlsanity.com/paper/2407.11351/) (2024)
- [HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection](https://mlsanity.com/paper/2301.03036/) (2023)
- [Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentation](https://mlsanity.com/paper/2407.11344/) (2024)
- [Alignment-Free RGBT Salient Object Detection: Semantics-guided Asymmetric Correlation Network and A Unified Benchmark](https://mlsanity.com/paper/2406.00917/) (2024)
- [Learning Adaptive Fusion Bank for Multi-modal Salient Object Detection](https://mlsanity.com/paper/2406.01127/) (2024)
- [Spider: A Unified Framework for Context-dependent Concept Segmentation](https://mlsanity.com/paper/2405.01002/) (2024)
- [Open-Vocabulary Camouflaged Object Segmentation](https://mlsanity.com/paper/2311.11241/) (2023)
- [MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selection](https://mlsanity.com/paper/2412.16876/) (2024)
- [Isomer: Isomerous Transformer for Zero-shot Video Object Segmentation](https://mlsanity.com/paper/2308.06693/) (2023)
- [Unified-modal Salient Object Detection via Adaptive Prompt Learning](https://mlsanity.com/paper/2311.16835/) (2023)
- [Learning Multi-view Anomaly Detection with Efficient Adaptive Selection](https://mlsanity.com/paper/2407.11935/) (2024)
- [Reducing Unimodal Bias in Multi-Modal Semantic Segmentation with Multi-Scale Functional Entropy Regularization](https://mlsanity.com/paper/2505.06635/) (2025)
- [Salient Object Detection in RGB-D Videos](https://mlsanity.com/paper/2310.15482/) (2023)
- [SSFam: Scribble Supervised Salient Object Detection Family](https://mlsanity.com/paper/2409.04817/) (2024)
- [MambaSOD: Dual Mamba-Driven Cross-Modal Fusion Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2410.15015/) (2024)
- [Lightweight RGB-D Salient Object Detection from a Speed-Accuracy Tradeoff Perspective](https://mlsanity.com/paper/2505.04758/) (2025)
- [Multispectral State-Space Feature Fusion: Bridging Shared and Cross-Parametric Interactions for Object Detection](https://mlsanity.com/paper/2507.14643/) (2025)
- [Unified Unsupervised Salient Object Detection via Knowledge Transfer](https://mlsanity.com/paper/2404.14759/) (2024)
- [KAN-SAM: Kolmogorov-Arnold Network Guided Segment Anything Model for RGB-T Salient Object Detection](https://mlsanity.com/paper/2504.05878/) (2025)
- [Mirror Complementary Transformer Network for RGB-thermal Salient Object Detection](https://mlsanity.com/paper/2207.03558/) (2022)
- [ComPtr: Towards Diverse Bi-source Dense Prediction Tasks via A Simple yet General Complementary Transformer](https://mlsanity.com/paper/2307.12349/) (2023)
- [All in One: RGB, RGB-D, and RGB-T Salient Object Detection](https://mlsanity.com/paper/2311.14746/) (2023)
- [CoLA: Conditional Dropout and Language-driven Robust Dual-modal Salient Object Detection](https://mlsanity.com/paper/2407.06780/) (2024)
- [UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation](https://mlsanity.com/paper/2509.16170/) (2025)
- [SAMSOD: Rethinking SAM Optimization for RGB-T Salient Object Detection](https://mlsanity.com/paper/2510.03689/) (2025)
- [Boosting Segment Anything Model to Generalize Visually Non-Salient Scenarios](https://mlsanity.com/paper/2601.00537/) (2026)
- [Salient Object Detection From Arbitrary Modalities](https://mlsanity.com/paper/2405.03352/) (2024)
- [Deep Fourier-embedded Network for RGB and Thermal Salient Object Detection](https://mlsanity.com/paper/2411.18409/) (2024)

## Abstract (arXiv)

Most of the existing bi-modal (RGB-D and RGB-T) salient object detection methods utilize the convolution operation and construct complex interweave fusion structures to achieve cross-modal information integration. The inherent local connectivity of the convolution operation constrains the performance of the convolution-based methods to a ceiling. In this work, we rethink these tasks from the perspective of global information alignment and transformation. Specifically, the proposed underlinecross-modunderlineal underlineview-mixed transformunderlineer (CAVER) cascades several cross-modal integration units to construct a top-down transformer-based information propagation path. CAVER treats the multi-scale and multi-modal feature integration as a sequence-to-sequence context propagation and update process built on a novel view-mixed attention mechanism. Besides, considering the quadratic complexity w.r.t. the number of input tokens, we design a parameter-free patch-wise token re-embedding strategy to simplify operations. Extensive experimental results on RGB-D and RGB-T SOD datasets demonstrate that such a simple two-stream encoder-decoder framework can surpass recent state-of-the-art methods when it is equipped with the proposed components. Code and pretrained models will be available at hrefhttps://github.com/lartpang/CAVERthe link.