# Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction

arXiv: https://arxiv.org/abs/2107.02630 · HTML: https://mlsanity.com/paper/2107.02630/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 62 references that are in the map.
- First posted on arXiv 6 Jul 2021.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Hyperspectral pansharpening: a review](https://mlsanity.com/paper/1504.04531/) (2015)
- [KiU-Net: Towards Accurate Segmentation of Biomedical Images using Over-complete Representations](https://mlsanity.com/paper/2006.04878/) (2020)
- [A convex formulation for hyperspectral image superresolution via subspace-based regularization](https://mlsanity.com/paper/1411.4005/) (2014)
- [Deep Hyperspectral Prior: Denoising, Inpainting, Super-Resolution](https://mlsanity.com/paper/1902.00301/) (2019)
- [KiU-Net: Overcomplete Convolutional Architectures for Biomedical Image and Volumetric Segmentation](https://mlsanity.com/paper/2010.01663/) (2020)
- [Over-and-Under Complete Convolutional RNN for MRI Reconstruction](https://mlsanity.com/paper/2106.08886/) (2021)
- [Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation](https://mlsanity.com/paper/1502.03121/) (2015)
- [Hyperspectral and Multispectral Image Fusion based on a Sparse Representation](https://mlsanity.com/paper/1409.5729/) (2014)
- [Bayesian Fusion of Multi-Band Images](https://mlsanity.com/paper/1307.5996/) (2013)
- [Exploring Overcomplete Representations for Single Image Deraining using CNNs](https://mlsanity.com/paper/2010.10661/) (2020)
- [Overcomplete Deep Subspace Clustering Networks](https://mlsanity.com/paper/2011.08306/) (2020)
- [Overcomplete Representations Against Adversarial Videos](https://mlsanity.com/paper/2012.04262/) (2020)

## Built on by (most cited first, 12 of 12)

- [Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review](https://mlsanity.com/paper/2205.01380/) (2022)
- [HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening](https://mlsanity.com/paper/2203.02503/) (2022)
- [Unsupervised Hyperspectral Pansharpening via Low-rank Diffusion Model](https://mlsanity.com/paper/2305.10925/) (2023)
- [SPIN Road Mapper: Extracting Roads from Aerial Images via Spatial and Interaction Space Graph Reasoning for Autonomous Driving](https://mlsanity.com/paper/2109.07701/) (2021)
- [Hyperspectral Pansharpening: Critical Review, Tools and Future Perspectives](https://mlsanity.com/paper/2407.01355/) (2024)
- [Band-wise Hyperspectral Image Pansharpening using CNN Model Propagation](https://mlsanity.com/paper/2311.06510/) (2023)
- [Real HSI-MSI-PAN image dataset for the hyperspectral/multi-spectral/panchromatic image fusion and super-resolution fields](https://mlsanity.com/paper/2407.02387/) (2024)
- [Zero-Shot Hyperspectral Pansharpening Using Hysteresis-Based Tuning for Spectral Quality Control](https://mlsanity.com/paper/2505.16658/) (2025)
- [Learning to Pan-sharpening with Memories of Spatial Details](https://mlsanity.com/paper/2306.16181/) (2023)
- [THAT: Token-wise High-frequency Augmentation Transformer for Hyperspectral Pansharpening](https://mlsanity.com/paper/2508.08183/) (2025)
- [PHASE: Physiology-Aware Hyperspectral Reconstruction via Object-to-Human Domain Adaptation](https://mlsanity.com/paper/2511.13020/) (2025)
- [CGFformer: Cluster-Guidance Frequency Transformer for Pansharpening](https://mlsanity.com/paper/2605.01490/) (2026)

## Abstract (arXiv)

Hyperspectral pansharpening aims to synthesize a low-resolution hyperspectral image (LR-HSI) with a registered panchromatic image (PAN) to generate an enhanced HSI with high spectral and spatial resolution. Recently proposed HS pansharpening methods have obtained remarkable results using deep convolutional networks (ConvNets), which typically consist of three steps: (1) up-sampling the LR-HSI, (2) predicting the residual image via a ConvNet, and (3) obtaining the final fused HSI by adding the outputs from first and second steps. Recent methods have leveraged Deep Image Prior (DIP) to up-sample the LR-HSI due to its excellent ability to preserve both spatial and spectral information, without learning from large data sets. However, we observed that the quality of up-sampled HSIs can be further improved by introducing an additional spatial-domain constraint to the conventional spectral-domain energy function. We define our spatial-domain constraint as the L1 distance between the predicted PAN image and the actual PAN image. To estimate the PAN image of the up-sampled HSI, we also propose a learnable spectral response function (SRF). Moreover, we noticed that the residual image between the up-sampled HSI and the reference HSI mainly consists of edge information and very fine structures. In order to accurately estimate fine information, we propose a novel over-complete network, called HyperKite, which focuses on learning high-level features by constraining the receptive from increasing in the deep layers. We perform experiments on three HSI datasets to demonstrate the superiority of our DIP-HyperKite over the state-of-the-art pansharpening methods. The deployment codes, pre-trained models, and final fusion outputs of our DIP-HyperKite and the methods used for the comparisons will be publicly made available at https://github.com/wgcban/DIP-HyperKite.git.