# CrossFuse: A Novel Cross Attention Mechanism based Infrared and Visible Image Fusion Approach

arXiv: https://arxiv.org/abs/2406.10581 · HTML: https://mlsanity.com/paper/2406.10581/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 68 references that are in the map.
- First posted on arXiv 15 Jun 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [DenseFuse: A Fusion Approach to Infrared and Visible Images](https://mlsanity.com/paper/1804.08361/) (2018)
- [DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion](https://mlsanity.com/paper/2303.06840/) (2023)
- [EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications](https://mlsanity.com/paper/2206.10589/) (2022)
- [RFN-Nest: An end-to-end residual fusion network for infrared and visible images](https://mlsanity.com/paper/2103.04286/) (2021)
- [LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images](https://mlsanity.com/paper/2304.05172/) (2023)
- [Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identification](https://mlsanity.com/paper/2205.02151/) (2022)
- [DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2003.09210/) (2020)
- [An Interactively Reinforced Paradigm for Joint Infrared-Visible Image Fusion and Saliency Object Detection](https://mlsanity.com/paper/2305.09999/) (2023)
- [A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognition](https://mlsanity.com/paper/2203.14779/) (2022)
- [Infrared and Visible Image Fusion using a Deep Learning Framework](https://mlsanity.com/paper/1804.06992/) (2018)
- [TransMEF: A Transformer-Based Multi-Exposure Image Fusion Framework using Self-Supervised Multi-Task Learning](https://mlsanity.com/paper/2112.01030/) (2021)
- [Image Fusion Transformer](https://mlsanity.com/paper/2107.09011/) (2021)
- [Multi-focus Image Fusion using dictionary learning and Low-Rank Representation](https://mlsanity.com/paper/1804.08355/) (2018)
- [Infrared and Visible Image Fusion with ResNet and zero-phase component analysis](https://mlsanity.com/paper/1806.07119/) (2018)
- [Bayesian Fusion for Infrared and Visible Images](https://mlsanity.com/paper/2005.05839/) (2020)
- [Multi-Modal Perception Attention Network with Self-Supervised Learning for Audio-Visual Speaker Tracking](https://mlsanity.com/paper/2112.07423/) (2021)
- [Visual Object Tracking on Multi-modal RGB-D Videos: A Review](https://mlsanity.com/paper/2201.09207/) (2022)

## Built on by (most cited first, 30 of 38)

- [One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion](https://mlsanity.com/paper/2502.19854/) (2025)
- [SMLNet: A SPD Manifold Learning Network for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2411.10679/) (2024)
- [EarthScape: A Multimodal Dataset for Surficial Geologic Mapping and Earth Surface Analysis](https://mlsanity.com/paper/2503.15625/) (2025)
- [Heart2Mind: Human-Centered Contestable Psychiatric Disorder Diagnosis System using Wearable ECG Monitors](https://mlsanity.com/paper/2505.11612/) (2025)
- [YOLOv11-RGBT: Towards a Comprehensive Single-Stage Multispectral Object Detection Framework](https://mlsanity.com/paper/2506.14696/) (2025)
- [FS-Diff: Semantic guidance and clarity-aware simultaneous multimodal image fusion and super-resolution](https://mlsanity.com/paper/2509.09427/) (2025)
- [Revisiting Generative Infrared and Visible Image Fusion Based on Human Cognitive Laws](https://mlsanity.com/paper/2510.26268/) (2025)
- [l0-Regularized Sparse Coding-based Interpretable Network for Multi-Modal Image Fusion](https://mlsanity.com/paper/2411.04519/) (2024)
- [Advances in Artificial Intelligence: A Review for the Creative Industries](https://mlsanity.com/paper/2501.02725/) (2025)
- [OCCO: LVM-guided Infrared and Visible Image Fusion Framework based on Object-aware and Contextual COntrastive Learning](https://mlsanity.com/paper/2503.18635/) (2025)
- [GrFormer: A Novel Transformer on Grassmann Manifold for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2506.14384/) (2025)
- [Towards Robust Optical-SAR Object Detection under Missing Modalities: A Dynamic Quality-Aware Fusion Framework](https://mlsanity.com/paper/2512.22447/) (2025)
- [FusionRegister: Every Infrared and Visible Image Fusion Deserves Registration](https://mlsanity.com/paper/2603.07667/) (2026)
- [Human Centric General Physical Intelligence for Agile Manufacturing Automation](https://mlsanity.com/paper/2508.11960/) (2025)
- [Text-Guided Channel Perturbation and Pretrained Knowledge Integration for Unified Multi-Modality Image Fusion](https://mlsanity.com/paper/2511.12432/) (2025)
- [VIIS: Visible and Infrared Information Synthesis for Severe Low-light Image Enhancement](https://mlsanity.com/paper/2412.13655/) (2024)
- [One Latent Space to Rule All Degradations: Unifying Restoration Knowledge for Image Fusion](https://mlsanity.com/paper/2503.07033/) (2025)
- [UniPTMs: The First Unified Multi-type PTM Site Prediction Model via Master-Slave Architecture-Based Multi-Stage Fusion Strategy and Hierarchical Contrastive Loss](https://mlsanity.com/paper/2506.05443/) (2025)
- [PCaM: A Progressive Focus Attention-Based Information Fusion Method for Improving Vision Transformer Domain Adaptation](https://mlsanity.com/paper/2506.17232/) (2025)
- [Knowledge Grafting: A Mechanism for Optimizing AI Model Deployment in Resource-Constrained Environments](https://mlsanity.com/paper/2507.19261/) (2025)
- [LSFDNet: A Single-Stage Fusion and Detection Network for Ships Using SWIR and LWIR](https://mlsanity.com/paper/2507.20574/) (2025)
- [UTA-Sign: Unsupervised Thermal Video Augmentation via Event-Assisted Traffic Signage Sketching](https://mlsanity.com/paper/2508.20594/) (2025)
- [JEPA-T: Joint-Embedding Predictive Architecture with Text Fusion for Image Generation](https://mlsanity.com/paper/2510.00974/) (2025)
- [WLC: Weber-Inspired Local Contrast Metric for Low-Altitude Image Fusion](https://mlsanity.com/paper/2512.15211/) (2025)
- [Multi-Grained Text-Guided Image Fusion for Multi-Exposure and Multi-Focus Scenarios](https://mlsanity.com/paper/2512.20556/) (2025)
- [Fusing in 3D: Free-Viewpoint Fusion Rendering with a 3D Infrared-Visible Scene Representation](https://mlsanity.com/paper/2601.12697/) (2026)
- [Fusion and Grouping Strategies in Deep Learning for Local Climate Zone Classification of Multimodal Remote Sensing Data](https://mlsanity.com/paper/2603.04562/) (2026)
- [Missing No More: Dictionary-Guided Cross-Modal Image Fusion under Missing Infrared](https://mlsanity.com/paper/2603.08018/) (2026)
- [RTFDNet: Fusion-Decoupling for Robust RGB-T Segmentation](https://mlsanity.com/paper/2603.09149/) (2026)
- [Beyond Strict Pairing: Arbitrarily Paired Training for High-Performance Infrared and Visible Image Fusion](https://mlsanity.com/paper/2603.21820/) (2026)

## Abstract (arXiv)

Multimodal visual information fusion aims to integrate the multi-sensor data into a single image which contains more complementary information and less redundant features. However the complementary information is hard to extract, especially for infrared and visible images which contain big similarity gap between these two modalities. The common cross attention modules only consider the correlation, on the contrary, image fusion tasks need focus on complementarity (uncorrelation). Hence, in this paper, a novel cross attention mechanism (CAM) is proposed to enhance the complementary information. Furthermore, a two-stage training strategy based fusion scheme is presented to generate the fused images. For the first stage, two auto-encoder networks with same architecture are trained for each modality. Then, with the fixed encoders, the CAM and a decoder are trained in the second stage. With the trained CAM, features extracted from two modalities are integrated into one fused feature in which the complementary information is enhanced and the redundant features are reduced. Finally, the fused image can be generated by the trained decoder. The experimental results illustrate that our proposed fusion method obtains the SOTA fusion performance compared with the existing fusion networks. The codes are available at https://github.com/hli1221/CrossFuse