# Learning a Graph Neural Network with Cross Modality Interaction for Image Fusion

arXiv: https://arxiv.org/abs/2308.03256 · HTML: https://mlsanity.com/paper/2308.03256/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 50 references that are in the map.
- First posted on arXiv 7 Aug 2023.

## Builds on (most cited first, 11 of 11)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [DenseFuse: A Fusion Approach to Infrared and Visible Images](https://mlsanity.com/paper/1804.08361/) (2018)
- [Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning](https://mlsanity.com/paper/2003.02437/) (2020)
- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2003.09210/) (2020)
- [Cascade Graph Neural Networks for RGB-D Salient Object Detection](https://mlsanity.com/paper/2008.03087/) (2020)

## Built on by (most cited first, 28 of 28)

- [Fusion-Mamba for Cross-modality Object Detection](https://mlsanity.com/paper/2404.09146/) (2024)
- [Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption](https://mlsanity.com/paper/2501.10761/) (2025)
- [All-weather Multi-Modality Image Fusion: Unified Framework and 100k Benchmark](https://mlsanity.com/paper/2402.02090/) (2024)
- [WaveMamba: Wavelet-Driven Mamba Fusion for RGB-Infrared Object Detection](https://mlsanity.com/paper/2507.18173/) (2025)
- [TSJNet: A Multi-modality Target and Semantic Awareness Joint-driven Image Fusion Network](https://mlsanity.com/paper/2402.01212/) (2024)
- [Multispectral State-Space Feature Fusion: Bridging Shared and Cross-Parametric Interactions for Object Detection](https://mlsanity.com/paper/2507.14643/) (2025)
- [TaGAT: Topology-Aware Graph Attention Network For Multi-modal Retinal Image Fusion](https://mlsanity.com/paper/2407.14188/) (2024)
- [Residual Prior-driven Frequency-aware Network for Image Fusion](https://mlsanity.com/paper/2507.06735/) (2025)
- [Bridging Human Evaluation to Infrared and Visible Image Fusion](https://mlsanity.com/paper/2603.03871/) (2026)
- [AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception](https://mlsanity.com/paper/2508.16881/) (2025)
- [FS-Diff: Semantic guidance and clarity-aware simultaneous multimodal image fusion and super-resolution](https://mlsanity.com/paper/2509.09427/) (2025)
- [MMA-UNet: A Multi-Modal Asymmetric UNet Architecture for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2404.17747/) (2024)
- [IRDFusion: Iterative Relation-Map Difference guided Feature Fusion for Multispectral Object Detection](https://mlsanity.com/paper/2509.09085/) (2025)
- [EvaNet: Towards More Efficient and Consistent Infrared and Visible Image Fusion Assessment](https://mlsanity.com/paper/2604.02896/) (2026)
- [A Semantic-Aware and Multi-Guided Network for Infrared-Visible Image Fusion](https://mlsanity.com/paper/2407.06159/) (2024)
- [textrmA^textrm2RNet: Adversarial Attack Resilient Network for Robust Infrared and Visible Image Fusion](https://mlsanity.com/paper/2412.09954/) (2024)
- [AlignFreeNet: Is Cross-Modal Pre-Alignment Necessary? An End-to-End Alignment-Free Lightweight Network for Visible-Infrared Object Detection](https://mlsanity.com/paper/2507.20146/) (2025)
- [Text-Guided Channel Perturbation and Pretrained Knowledge Integration for Unified Multi-Modality Image Fusion](https://mlsanity.com/paper/2511.12432/) (2025)
- [Multi-Modal Image Fusion via Intervention-Stable Feature Learning](https://mlsanity.com/paper/2603.23272/) (2026)
- [LSFDNet: A Single-Stage Fusion and Detection Network for Ships Using SWIR and LWIR](https://mlsanity.com/paper/2507.20574/) (2025)
- [SGDFuse: SAM-Guided Diffusion Model for High-Fidelity Infrared and Visible Image Fusion](https://mlsanity.com/paper/2508.05264/) (2025)
- [Bridging Multimodal Fusion and Expert Routing via Spectral Reliability Descriptors for Robust Object Detection](https://mlsanity.com/paper/2606.01173/) (2026)
- [Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction](https://mlsanity.com/paper/2606.26812/) (2026)
- [Dual Sparse Aggregation Transformer for Multispectral Object Detection](https://mlsanity.com/paper/2606.31015/) (2026)
- [InfraNet: Quality-Aware RGB Guidance for Efficient Infrared Object Detection](https://mlsanity.com/paper/2607.03795/) (2026)
- [Decoupling Cross-Modality Manifold Discrepancy: Leveraging Visible Diffusion Priors for Infrared Super-Resolution](https://mlsanity.com/paper/2607.21174/) (2026)
- [DRPFNet: Dual-domain Residual Progressive Fusion Network for RGB-Thermal Object Detection](https://mlsanity.com/paper/2608.03370/) (2026)
- [RegisterBridgeMM: A Register-Centric Framework for RGB-Infrared Object Detection](https://mlsanity.com/paper/2608.04833/) (2026)

## Abstract (arXiv)

Infrared and visible image fusion has gradually proved to be a vital fork in the field of multi-modality imaging technologies. In recent developments, researchers not only focus on the quality of fused images but also evaluate their performance in downstream tasks. Nevertheless, the majority of methods seldom put their eyes on the mutual learning from different modalities, resulting in fused images lacking significant details and textures. To overcome this issue, we propose an interactive graph neural network (GNN)-based architecture between cross modality for fusion, called IGNet. Specifically, we first apply a multi-scale extractor to achieve shallow features, which are employed as the necessary input to build graph structures. Then, the graph interaction module can construct the extracted intermediate features of the infrared/visible branch into graph structures. Meanwhile, the graph structures of two branches interact for cross-modality and semantic learning, so that fused images can maintain the important feature expressions and enhance the performance of downstream tasks. Besides, the proposed leader nodes can improve information propagation in the same modality. Finally, we merge all graph features to get the fusion result. Extensive experiments on different datasets (TNO, MFNet and M3FD) demonstrate that our IGNet can generate visually appealing fused images while scoring averagely 2.59% mAP@.5 and 7.77% mIoU higher in detection and segmentation than the compared state-of-the-art methods. The source code of the proposed IGNet can be available at https://github.com/lok-18/IGNet.