# Spatial-Spectral Fusion by Combining Deep Learning and Variation Model

arXiv: https://arxiv.org/abs/1809.00764 · HTML: https://mlsanity.com/paper/1809.00764/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2018.

## Built on by (most cited first, 6 of 6)

- [Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review](https://mlsanity.com/paper/2205.01380/) (2022)
- [Coupling Model-Driven and Data-Driven Methods for Remote Sensing Image Restoration and Fusion](https://mlsanity.com/paper/2108.06073/) (2021)
- [Model Inspired Autoencoder for Unsupervised Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2110.11591/) (2021)
- [An Integrated Framework for the Heterogeneous Spatio-Spectral-Temporal Fusion of Remote Sensing Images](https://mlsanity.com/paper/2109.00400/) (2021)
- [CEU-Net: Ensemble Semantic Segmentation of Hyperspectral Images Using Clustering](https://mlsanity.com/paper/2203.04873/) (2022)
- [Degradation Alchemy: Self-Supervised Unknown-to-Known Transformation for Blind Hyperspectral Image Fusion](https://mlsanity.com/paper/2503.14892/) (2025)

## Abstract (arXiv)

In the field of spatial-spectral fusion, the model-based method and the deep learning (DL)-based method are state-of-the-art. This paper presents a fusion method that incorporates the deep neural network into the model-based method for the most common case in the spatial-spectral fusion: PAN/multispectral (MS) fusion. Specifically, we first map the gradient of the high spatial resolution panchromatic image (HR-PAN) and the low spatial resolution multispectral image (LR-MS) to the gradient of the high spatial resolution multispectral image (HR-MS) via a deep residual convolutional neural network (CNN). Then we construct a fusion framework by the LR-MS image, the gradient prior learned from the gradient network, and the ideal fused image. Finally, an iterative optimization algorithm is used to solve the fusion model. Both quantitative and visual assessments on high-quality images from various sources demonstrate that the proposed fusion method is superior to all the mainstream algorithms included in the comparison in terms of overall fusion accuracy.