Modality Dominance-Aware Optimization for Embodied RGB-Infrared Perception
Not on the map. Why?
Abstract (arXiv)
RGB-Infrared (RGB-IR) multimodal perception is fundamental to embodied multimedia systems operating in complex physical environments. Although recent cross-modal fusion methods have advanced RGB-IR detection, the optimization dynamics caused by asymmetric modality characteristics remain underexplored. In practice, disparities in information density and feature quality introduce persistent optimization bias, leading training to overemphasize a dominant modality and hindering effective fusion. To quantify this phenomenon, we propose the Modality Dominance Index (MDI), which measures modality dominance by jointly modeling feature entropy and gradient contribution. Based on MDI, we develop a Modality Dominance-Aware Cross-modal Learning (MDACL) framework that regulates cross-modal optimization. MDACL incorporates Hierarchical Cross-modal Guidance (HCG) to enhance feature alignment and Adversarial Equilibrium Regularization (AER) to balance optimization dynamics during fusion. Extensive experiments on three RGB-IR benchmarks demonstrate that MDACL effectively mitigates optimization bias and achieves SOTA performance.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 2 Jan 2026.
Builds on
17 of 25 references.
References · 17
- 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 344
- 2022Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detectioncited by 317
- 2021LLVIP: A Visible-infrared Paired Dataset for Low-light Visioncited by 231
- 2022CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusioncited by 223
- 2023Equivariant Multi-Modality Image Fusioncited by 99
- 2020Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blockscited by 85
- 2024Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusioncited by 84
- 2023ICAFusion: Iterative Cross-Attention Guided Feature Fusion for Multispectral Object Detectioncited by 76
- 2023Multimodal Representation Learning by Alternating Unimodal Adaptationcited by 75
- 2023Boosting Multi-modal Model Performance with Adaptive Gradient Modulationcited by 73
- 2024MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistancecited by 73
- 2024Fusion-Mamba for Cross-modality Object Detectioncited by 67
5 more
- 2023Enhancing multimodal cooperation via sample-level modality valuationcited by 44
- 2023MMCosine: Multi-Modal Cosine Loss Towards Balanced Audio-Visual Fine-Grained Learningcited by 28
- 2024DAMSDet: Dynamic Adaptive Multispectral Detection Transformer with Competitive Query Selection and Adaptive Feature Fusioncited by 28
- 2024UniRGB-IR: A Unified Framework for Visible-Infrared Semantic Tasks via Adapter Tuningcited by 17
- 2025LUT-Fuse: Towards Extremely Fast Infrared and Visible Image Fusion via Distillation to Learnable Look-Up Tablescited by 9
Built on by
Citing papers per year: 2026: 1.
1 paper.