# UMCFuse: A Unified Multiple Complex Scenes Infrared and Visible Image Fusion Framework

arXiv: https://arxiv.org/abs/2402.02096 · HTML: https://mlsanity.com/paper/2402.02096/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 103 references that are in the map.
- First posted on arXiv 3 Feb 2024.

## Builds on (most cited first, 27 of 27)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [RFN-Nest: An end-to-end residual fusion network for infrared and visible images](https://mlsanity.com/paper/2103.04286/) (2021)
- [NH-HAZE: An Image Dehazing Benchmark with Non-Homogeneous Hazy and Haze-Free Images](https://mlsanity.com/paper/2005.03560/) (2020)
- [LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images](https://mlsanity.com/paper/2304.05172/) (2023)
- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2303.08600/) (2023)
- [CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion](https://mlsanity.com/paper/2211.10960/) (2022)
- [MDLatLRR: A novel decomposition method for infrared and visible image fusion](https://mlsanity.com/paper/1811.02291/) (2018)
- [Cascade Graph Neural Networks for RGB-D Salient Object Detection](https://mlsanity.com/paper/2008.03087/) (2020)
- [SCADE: NeRFs from Space Carving with Ambiguity-Aware Depth Estimates](https://mlsanity.com/paper/2303.13582/) (2023)
- [TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network](https://mlsanity.com/paper/2201.10147/) (2022)
- [PAIF: Perception-Aware Infrared-Visible Image Fusion for Attack-Tolerant Semantic Segmentation](https://mlsanity.com/paper/2308.03979/) (2023)
- [NLH: A Blind Pixel-level Non-local Method for Real-world Image Denoising](https://mlsanity.com/paper/1906.06834/) (2019)
- [Bayesian Fusion for Infrared and Visible Images](https://mlsanity.com/paper/2005.05839/) (2020)
- [Bridging the Gap between Multi-focus and Multi-modal: A Focused Integration Framework for Multi-modal Image Fusion](https://mlsanity.com/paper/2311.01886/) (2023)
- [TSJNet: A Multi-modality Target and Semantic Awareness Joint-driven Image Fusion Network](https://mlsanity.com/paper/2402.01212/) (2024)
- [SAMF: Small-Area-Aware Multi-focus Image Fusion for Object Detection](https://mlsanity.com/paper/2401.08357/) (2024)
- [Generation and Recombination for Multifocus Image Fusion with Free Number of Inputs](https://mlsanity.com/paper/2309.04657/) (2023)
- [Interactive Feature Embedding for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2211.04877/) (2022)
- [Infrared and visible image fusion based on Multi-State Contextual Hidden Markov Model](https://mlsanity.com/paper/2201.10739/) (2022)

## Built on by (most cited first, 9 of 9)

- [Shuffle Mamba: State Space Models with Random Shuffle for Multi-Modal Image Fusion](https://mlsanity.com/paper/2409.01728/) (2024)
- [AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception](https://mlsanity.com/paper/2508.16881/) (2025)
- [Text-Guided Channel Perturbation and Pretrained Knowledge Integration for Unified Multi-Modality Image Fusion](https://mlsanity.com/paper/2511.12432/) (2025)
- [Multi-Modal Image Fusion via Intervention-Stable Feature Learning](https://mlsanity.com/paper/2603.23272/) (2026)
- [CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration](https://mlsanity.com/paper/2603.02560/) (2026)
- [FTPFusion: Frequency-Aware Infrared and Visible Video Fusion with Temporal Perturbation](https://mlsanity.com/paper/2604.01900/) (2026)
- [MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction](https://mlsanity.com/paper/2604.01958/) (2026)
- [Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction](https://mlsanity.com/paper/2606.26812/) (2026)
- [RoES: Rotational Equivariant Selective-frequency Fusion for Multimodal Images](https://mlsanity.com/paper/2609.12497/) (2026)

## Abstract (arXiv)

Infrared and visible image fusion has emerged as a prominent research area in computer vision. However, little attention has been paid to the fusion task in complex scenes, leading to sub-optimal results under interference. To fill this gap, we propose a unified framework for infrared and visible images fusion in complex scenes, termed UMCFuse. Specifically, we classify the pixels of visible images from the degree of scattering of light transmission, allowing us to separate fine details from overall intensity. Maintaining a balance between interference removal and detail preservation is essential for the generalization capacity of the proposed method. Therefore, we propose an adaptive denoising strategy for the fusion of detail layers. Meanwhile, we fuse the energy features from different modalities by analyzing them from multiple directions. Extensive fusion experiments on real and synthetic complex scenes datasets cover adverse weather conditions, noise, blur, overexposure, fire, as well as downstream tasks including semantic segmentation, object detection, salient object detection, and depth estimation, consistently indicate the superiority of the proposed method compared with the recent representative methods. Our code is available at https://github.com/ixilai/UMCFuse.