# Unsupervised Hyperspectral and Multispectral Images Fusion Based on the Cycle Consistency

arXiv: https://arxiv.org/abs/2307.03413 · HTML: https://mlsanity.com/paper/2307.03413/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 47 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Hyperspectral pansharpening: a review](https://mlsanity.com/paper/1504.04531/) (2015)
- [A convex formulation for hyperspectral image superresolution via subspace-based regularization](https://mlsanity.com/paper/1411.4005/) (2014)
- [Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net](https://mlsanity.com/paper/1901.03281/) (2019)
- [Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution](https://mlsanity.com/paper/2007.05230/) (2020)
- [Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/1804.05042/) (2018)
- [Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation](https://mlsanity.com/paper/1502.03121/) (2015)
- [Hyperspectral and Multispectral Image Fusion based on a Sparse Representation](https://mlsanity.com/paper/1409.5729/) (2014)
- [Recent Advances and New Guidelines on Hyperspectral and Multispectral Image Fusion](https://mlsanity.com/paper/2008.03426/) (2020)
- [Guided Deep Decoder: Unsupervised Image Pair Fusion](https://mlsanity.com/paper/2007.11766/) (2020)
- [Hyperspectral Image Super-Resolution via Deep Prior Regularization with Parameter Estimation](https://mlsanity.com/paper/2009.04237/) (2020)

## Abstract (arXiv)

Hyperspectral images (HSI) with abundant spectral information reflected materials property usually perform low spatial resolution due to the hardware limits. Meanwhile, multispectral images (MSI), e.g., RGB images, have a high spatial resolution but deficient spectral signatures. Hyperspectral and multispectral image fusion can be cost-effective and efficient for acquiring both high spatial resolution and high spectral resolution images. Many of the conventional HSI and MSI fusion algorithms rely on known spatial degradation parameters, i.e., point spread function, spectral degradation parameters, spectral response function, or both of them. Another class of deep learning-based models relies on the ground truth of high spatial resolution HSI and needs large amounts of paired training images when working in a supervised manner. Both of these models are limited in practical fusion scenarios. In this paper, we propose an unsupervised HSI and MSI fusion model based on the cycle consistency, called CycFusion. The CycFusion learns the domain transformation between low spatial resolution HSI (LrHSI) and high spatial resolution MSI (HrMSI), and the desired high spatial resolution HSI (HrHSI) are considered to be intermediate feature maps in the transformation networks. The CycFusion can be trained with the objective functions of marginal matching in single transform and cycle consistency in double transforms. Moreover, the estimated PSF and SRF are embedded in the model as the pre-training weights, which further enhances the practicality of our proposed model. Experiments conducted on several datasets show that our proposed model outperforms all compared unsupervised fusion methods. The codes of this paper will be available at this address: https: //github.com/shuaikaishi/CycFusion for reproducibility.