Bridging the RGB-IR Gap: Consensus and Discrepancy Modeling for Text-Guided Multispectral Detection
Not on the map. Why?
Abstract (arXiv)
Text-guided multispectral object detection uses text semantics to guide semantic-aware cross-modal interaction between RGB and IR for more robust perception. However, notable limitations remain: (1) existing methods often use text only as an auxiliary semantic enhancement signal, without exploiting its guiding role to bridge the inherent granularity asymmetry between RGB and IR; and (2) conventional data-driven attention-based fusion tends to emphasize stable consensus while overlooking potentially valuable cross-modal discrepancies. To address these issues, we propose a semantic bridge fusion framework with bi-support modeling for multispectral object detection. Specifically, text is used as a shared semantic bridge to align RGB and IR responses under a unified category condition, while the recalibrated thermal semantic prior is projected onto the RGB branch for semantic-level mapping fusion. We further formulate RGB-IR interaction evidence into the regular consensus support and the complementary discrepancy support that contains potentially discriminative cues, and introduce them into fusion via dynamic recalibration as a structured inductive bias. In addition, we design a bidirectional semantic alignment module for closed-loop vision-text guidance enhancement. Extensive experiments demonstrate the effectiveness of the proposed fusion framework and its superior detection performance on multispectral benchmarks. Code is available at https://github.com/zhenwang5372/Bridging-RGB-IR-Gap.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 59 references that are in the map. First posted on arXiv 13 Apr 2026.
Builds on
21 of 59 references.
References · 21
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
- 2022Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detectioncited by 317
- 2021ReDet: A Rotation-equivariant Detector for Aerial Object Detectioncited by 203
- 2020Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learningcited by 152
- 2021Cross-Modality Fusion Transformer for Multispectral Object Detectioncited by 86
- 2020Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blockscited by 85
- 2024Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusioncited by 84
- 2020Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problemscited by 83
- 2023Dense Distinct Query for End-to-End Object Detectioncited by 77
- 2023ICAFusion: Iterative Cross-Attention Guided Feature Fusion for Multispectral Object Detectioncited by 76
9 more
- 2018Illumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detectioncited by 74
- 2024Fusion-Mamba for Cross-modality Object Detectioncited by 67
- 2023C2Former: Calibrated and Complementary Transformer for RGB-Infrared Object Detectioncited by 58
- 2025Infrared and Visible Image Fusion: From Data Compatibility to Task Adaptioncited by 46
- 2024Removal then Selection: A Coarse-to-Fine Fusion Perspective for RGB-Infrared Object Detectioncited by 16
- 2025WaveMamba: Wavelet-Driven Mamba Fusion for RGB-Infrared Object Detectioncited by 14
- 2025Multispectral State-Space Feature Fusion: Bridging Shared and Cross-Parametric Interactions for Object Detectioncited by 6
- 2026SLGNet: Synergizing Structural Priors and Language-Guided Modulation for Multimodal Object Detectioncited by 4
- 2026Modality Dominance-Aware Optimization for Embodied RGB-Infrared Perceptioncited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.