# Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net

arXiv: https://arxiv.org/abs/1901.03281 · HTML: https://mlsanity.com/paper/1901.03281/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jan 2019.

## Built on by (most cited first, 30 of 35)

- [Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review](https://mlsanity.com/paper/2205.01380/) (2022)
- [Deep Gradient Projection Networks for Pan-sharpening](https://mlsanity.com/paper/2103.04584/) (2021)
- [Text Prior Guided Scene Text Image Super-resolution](https://mlsanity.com/paper/2106.15368/) (2021)
- [Unsupervised Hyperspectral Image Super-Resolution via Self-Supervised Modality Decoupling](https://mlsanity.com/paper/2412.04802/) (2024)
- [InDuDoNet+: A Deep Unfolding Dual Domain Network for Metal Artifact Reduction in CT Images](https://mlsanity.com/paper/2112.12660/) (2021)
- [Decoupled-and-Coupled Networks: Self-Supervised Hyperspectral Image Super-Resolution with Subpixel Fusion](https://mlsanity.com/paper/2205.03742/) (2022)
- [KXNet: A Model-Driven Deep Neural Network for Blind Super-Resolution](https://mlsanity.com/paper/2209.10305/) (2022)
- [Adaptive Convolutional Dictionary Network for CT Metal Artifact Reduction](https://mlsanity.com/paper/2205.07471/) (2022)
- [PanFlowNet: A Flow-Based Deep Network for Pan-sharpening](https://mlsanity.com/paper/2305.07774/) (2023)
- [Dual-Stage Approach Toward Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2204.04387/) (2022)
- [Hyperspectral Image Super Resolution with Real Unaligned RGB Guidance](https://mlsanity.com/paper/2302.06298/) (2023)
- [RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining](https://mlsanity.com/paper/2107.06808/) (2021)
- [Coupling Model-Driven and Data-Driven Methods for Remote Sensing Image Restoration and Fusion](https://mlsanity.com/paper/2108.06073/) (2021)
- [Model Inspired Autoencoder for Unsupervised Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2110.11591/) (2021)
- [Deep Posterior Distribution-based Embedding for Hyperspectral Image Super-resolution](https://mlsanity.com/paper/2205.14887/) (2022)
- [Deep Amended Gradient Descent for Efficient Spectral Reconstruction from Single RGB Images](https://mlsanity.com/paper/2108.05547/) (2021)
- [Local-Global Transformer Enhanced Unfolding Network for Pan-sharpening](https://mlsanity.com/paper/2304.14612/) (2023)
- [Learned Image Reasoning Prior Penetrates Deep Unfolding Network for Panchromatic and Multi-Spectral Image Fusion](https://mlsanity.com/paper/2308.16083/) (2023)
- [Limitations of Data-Driven Spectral Reconstruction -- An Optics-Aware Analysis](https://mlsanity.com/paper/2401.03835/) (2024)
- [Probability-based Global Cross-modal Upsampling for Pansharpening](https://mlsanity.com/paper/2303.13659/) (2023)
- [Hyperspectral and Multispectral Image Fusion Using the Conditional Denoising Diffusion Probabilistic Model](https://mlsanity.com/paper/2307.03423/) (2023)
- [Hyperspectral Image Super-resolution with Deep Priors and Degradation Model Inversion](https://mlsanity.com/paper/2201.09851/) (2022)
- [RASMD: RGB And SWIR Multispectral Driving Dataset for Robust Perception in Adverse Conditions](https://mlsanity.com/paper/2504.07603/) (2025)
- [Enhanced Hyperspectral Image Super-Resolution via RGB Fusion and TV-TV Minimization](https://mlsanity.com/paper/2106.07066/) (2021)
- [LD-GAN: Low-Dimensional Generative Adversarial Network for Spectral Image Generation with Variance Regularization](https://mlsanity.com/paper/2305.00132/) (2023)
- [Explainable AI for Earth Observation: Current Methods, Open Challenges, and Opportunities](https://mlsanity.com/paper/2311.04491/) (2023)
- [Physics-Inspired Degradation Models for Hyperspectral Image Fusion](https://mlsanity.com/paper/2402.02411/) (2024)
- [Test-time Training for Hyperspectral Image Super-resolution](https://mlsanity.com/paper/2409.08667/) (2024)
- [From Less to More: Spectral Splitting and Aggregation Network for Hyperspectral Face Super-Resolution](https://mlsanity.com/paper/2108.13584/) (2021)
- [Applications of Explainable artificial intelligence in Earth system science](https://mlsanity.com/paper/2406.11882/) (2024)

## Abstract (arXiv)

Hyperspectral imaging can help better understand the characteristics of different materials, compared with traditional image systems. However, only high-resolution multispectral (HrMS) and low-resolution hyperspectral (LrHS) images can generally be captured at video rate in practice. In this paper, we propose a model-based deep learning approach for merging an HrMS and LrHS images to generate a high-resolution hyperspectral (HrHS) image. In specific, we construct a novel MS/HS fusion model which takes the observation models of low-resolution images and the low-rankness knowledge along the spectral mode of HrHS image into consideration. Then we design an iterative algorithm to solve the model by exploiting the proximal gradient method. And then, by unfolding the designed algorithm, we construct a deep network, called MS/HS Fusion Net, with learning the proximal operators and model parameters by convolutional neural networks. Experimental results on simulated and real data substantiate the superiority of our method both visually and quantitatively as compared with state-of-the-art methods along this line of research.