# Infrared and Visible Image Fusion with ResNet and zero-phase component analysis

arXiv: https://arxiv.org/abs/1806.07119 · HTML: https://mlsanity.com/paper/1806.07119/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2018.

## Built on by (most cited first, 16 of 16)

- [RFN-Nest: An end-to-end residual fusion network for infrared and visible images](https://mlsanity.com/paper/2103.04286/) (2021)
- [LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images](https://mlsanity.com/paper/2304.05172/) (2023)
- [CrossFuse: A Novel Cross Attention Mechanism based Infrared and Visible Image Fusion Approach](https://mlsanity.com/paper/2406.10581/) (2024)
- [TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network](https://mlsanity.com/paper/2201.10147/) (2022)
- [FCDFusion: a Fast, Low Color Deviation Method for Fusing Visible and Infrared Image Pairs](https://mlsanity.com/paper/2408.01080/) (2024)
- [Deep Learning for Spatio-Temporal Fusion in Land Surface Temperature Estimation: A Comprehensive Survey, Experimental Analysis, and Future Trends](https://mlsanity.com/paper/2412.16631/) (2024)
- [FusionVAE: A Deep Hierarchical Variational Autoencoder for RGB Image Fusion](https://mlsanity.com/paper/2209.11277/) (2022)
- [Infrared and visible image fusion based on Multi-State Contextual Hidden Markov Model](https://mlsanity.com/paper/2201.10739/) (2022)
- [FuseFormer: A Transformer for Visual and Thermal Image Fusion](https://mlsanity.com/paper/2402.00971/) (2024)
- [Visible and Infrared Image Fusion Using Encoder-Decoder Network](https://mlsanity.com/paper/2412.08073/) (2024)
- [Physics Driven Deep Retinex Fusion for Adaptive Infrared and Visible Image Fusion](https://mlsanity.com/paper/2112.02869/) (2021)
- [Unsupervised Image Fusion Method based on Feature Mutual Mapping](https://mlsanity.com/paper/2201.10152/) (2022)
- [Explicit and implicit models in infrared and visible image fusion](https://mlsanity.com/paper/2206.09581/) (2022)
- [An Attention-Guided and Wavelet-Constrained Generative Adversarial Network for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2210.11018/) (2022)
- [Fusion of Infrared and Visible Images based on Spatial-Channel Attentional Mechanism](https://mlsanity.com/paper/2308.13672/) (2023)
- [GAN-HA: A generative adversarial network with a novel heterogeneous dual-discriminator network and a new attention-based fusion strategy for infrared and visible image fusion](https://mlsanity.com/paper/2404.15992/) (2024)

## Abstract (arXiv)

Feature extraction and processing tasks play a key role in Image Fusion, and the fusion performance is directly affected by the different features and processing methods undertaken. By contrast, most of deep learning-based methods use deep features directly without feature extraction or processing. This leads to the fusion performance degradation in some cases. To solve these drawbacks, we propose a deep features and zero-phase component analysis (ZCA) based novel fusion framework is this paper. Firstly, the residual network (ResNet) is used to extract deep features from source images. Then ZCA is utilized to normalize the deep features and obtain initial weight maps. The final weight maps are obtained by employing a soft-max operation in association with the initial weight maps. Finally, the fused image is reconstructed using a weighted-averaging strategy. Compared with the existing fusion methods, experimental results demonstrate that the proposed framework achieves better performance in both objective assessment and visual quality. The code of our fusion algorithm is available at https://github.com/hli1221/imagefusion_resnet50