# Modality-Decoupled RGB-Thermal Object Detector via Query Fusion

arXiv: https://arxiv.org/abs/2601.08458 · HTML: https://mlsanity.com/paper/2601.08458/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 13 Jan 2026.

## Builds on (most cited first, 15 of 15)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [Cross-Modality Fusion Transformer for Multispectral Object Detection](https://mlsanity.com/paper/2111.00273/) (2021)
- [Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blocks](https://mlsanity.com/paper/2009.12664/) (2020)
- [Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems](https://mlsanity.com/paper/2008.03043/) (2020)
- [Multispectral Deep Neural Networks for Pedestrian Detection](https://mlsanity.com/paper/1611.02644/) (2016)
- [ICAFusion: Iterative Cross-Attention Guided Feature Fusion for Multispectral Object Detection](https://mlsanity.com/paper/2308.07504/) (2023)
- [Multimodal Object Detection via Probabilistic Ensembling](https://mlsanity.com/paper/2104.02904/) (2021)
- [Fusion-Mamba for Cross-modality Object Detection](https://mlsanity.com/paper/2404.09146/) (2024)
- [Cross-Modality Proposal-guided Feature Mining for Unregistered RGB-Thermal Pedestrian Detection](https://mlsanity.com/paper/2308.12111/) (2023)
- [Rethinking Early-Fusion Strategies for Improved Multispectral Object Detection](https://mlsanity.com/paper/2405.16038/) (2024)
- [Learning A Robust RGB-Thermal Detector for Extreme Modality Imbalance](https://mlsanity.com/paper/2505.22154/) (2025)

## Abstract (arXiv)

The advantage of RGB-Thermal (RGB-T) detection lies in its ability to perform modality fusion and integrate cross-modality complementary information, enabling robust detection under diverse illumination and weather conditions. However, under extreme conditions where one modality exhibits poor quality and disturbs detection, modality separation is necessary to mitigate the impact of noise. To address this problem, we propose a Modality-Decoupled RGB-T detection framework with Query Fusion (MDQF) to balance modality complementation and separation. In this framework, DETR-like detectors are employed as separate branches for the RGB and TIR images, with query fusion interspersed between the two branches in each refinement stage. Herein, query fusion is performed by feeding the high-quality queries from one branch to the other one after query selection and adaptation. This design effectively excludes the degraded modality and corrects the predictions using high-quality queries. Moreover, the decoupled framework allows us to optimize each individual branch with unpaired RGB or TIR images, eliminating the need for paired RGB-T data. Extensive experiments demonstrate that our approach delivers superior performance to existing RGB-T detectors and achieves better modality independence.