# Bridging the Gap between Multi-focus and Multi-modal: A Focused Integration Framework for Multi-modal Image Fusion

arXiv: https://arxiv.org/abs/2311.01886 · HTML: https://mlsanity.com/paper/2311.01886/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 53 references that are in the map.
- First posted on arXiv 3 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images](https://mlsanity.com/paper/2304.05172/) (2023)
- [MDLatLRR: A novel decomposition method for infrared and visible image fusion](https://mlsanity.com/paper/1811.02291/) (2018)
- [Towards Online Domain Adaptive Object Detection](https://mlsanity.com/paper/2204.05289/) (2022)
- [Plugging Self-Supervised Monocular Depth into Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2110.06685/) (2021)
- [Video Salient Object Detection via Contrastive Features and Attention Modules](https://mlsanity.com/paper/2111.02368/) (2021)
- [Interactive Feature Embedding for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2211.04877/) (2022)
- [Semi-Sparsity for Smoothing Filters](https://mlsanity.com/paper/2107.00627/) (2021)
- [Sparse Norm Filtering](https://mlsanity.com/paper/1305.3971/) (2013)

## Built on by (most cited first, 11 of 11)

- [TSJNet: A Multi-modality Target and Semantic Awareness Joint-driven Image Fusion Network](https://mlsanity.com/paper/2402.01212/) (2024)
- [UMCFuse: A Unified Multiple Complex Scenes Infrared and Visible Image Fusion Framework](https://mlsanity.com/paper/2402.02096/) (2024)
- [FlexiD-Fuse: Flexible number of inputs multi-modal medical image fusion based on diffusion model](https://mlsanity.com/paper/2509.09456/) (2025)
- [AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception](https://mlsanity.com/paper/2508.16881/) (2025)
- [MMA-UNet: A Multi-Modal Asymmetric UNet Architecture for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2404.17747/) (2024)
- [Text-Guided Channel Perturbation and Pretrained Knowledge Integration for Unified Multi-Modality Image Fusion](https://mlsanity.com/paper/2511.12432/) (2025)
- [Gradient-based multi-focus image fusion with focus-aware saliency enhancement](https://mlsanity.com/paper/2509.22392/) (2025)
- [A Luminance-Aware Multi-Scale Network for Polarization Image Fusion with a Multi-Scene Dataset](https://mlsanity.com/paper/2510.24379/) (2025)
- [FTPFusion: Frequency-Aware Infrared and Visible Video Fusion with Temporal Perturbation](https://mlsanity.com/paper/2604.01900/) (2026)
- [MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction](https://mlsanity.com/paper/2604.01958/) (2026)
- [Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction](https://mlsanity.com/paper/2606.26812/) (2026)

## Abstract (arXiv)

Multi-modal image fusion (MMIF) integrates valuable information from different modality images into a fused one. However, the fusion of multiple visible images with different focal regions and infrared images is a unprecedented challenge in real MMIF applications. This is because of the limited depth of the focus of visible optical lenses, which impedes the simultaneous capture of the focal information within the same scene. To address this issue, in this paper, we propose a MMIF framework for joint focused integration and modalities information extraction. Specifically, a semi-sparsity-based smoothing filter is introduced to decompose the images into structure and texture components. Subsequently, a novel multi-scale operator is proposed to fuse the texture components, capable of detecting significant information by considering the pixel focus attributes and relevant data from various modal images. Additionally, to achieve an effective capture of scene luminance and reasonable contrast maintenance, we consider the distribution of energy information in the structural components in terms of multi-directional frequency variance and information entropy. Extensive experiments on existing MMIF datasets, as well as the object detection and depth estimation tasks, consistently demonstrate that the proposed algorithm can surpass the state-of-the-art methods in visual perception and quantitative evaluation. The code is available at https://github.com/ixilai/MFIF-MMIF.