# Variational Zero-shot Multispectral Pansharpening

arXiv: https://arxiv.org/abs/2407.06633 · HTML: https://mlsanity.com/paper/2407.06633/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 90 references that are in the map.
- First posted on arXiv 9 Jul 2024.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Time-Dependent Deep Image Prior for Dynamic MRI](https://mlsanity.com/paper/1910.01684/) (2019)
- [Computed Tomography Reconstruction Using Deep Image Prior and Learned Reconstruction Methods](https://mlsanity.com/paper/2003.04989/) (2020)
- [A Multi-Scale and Multi-Depth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening](https://mlsanity.com/paper/1712.09809/) (2017)
- [Pansharpening via Detail Injection Based Convolutional Neural Networks](https://mlsanity.com/paper/1806.08898/) (2018)
- [PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening](https://mlsanity.com/paper/1805.03371/) (2018)
- [Boosting the accuracy of multi-spectral image pan-sharpening by learning a deep residual network](https://mlsanity.com/paper/1705.07556/) (2017)
- [Hyperspectral Denoising Using Unsupervised Disentangled Spatio-Spectral Deep Priors](https://mlsanity.com/paper/2102.12310/) (2021)
- [PGMAN: An Unsupervised Generative Multi-adversarial Network for Pan-sharpening](https://mlsanity.com/paper/2012.09054/) (2020)
- [Unsupervised Pansharpening Based on Self-Attention Mechanism](https://mlsanity.com/paper/2006.09303/) (2020)
- [Unsupervised Cycle-consistent Generative Adversarial Networks for Pan-sharpening](https://mlsanity.com/paper/2109.09395/) (2021)
- [LDP-Net: An Unsupervised Pansharpening Network Based on Learnable Degradation Processes](https://mlsanity.com/paper/2111.12483/) (2021)

## Built on by (most cited first, 5 of 5)

- [FusionRF: High-Fidelity Satellite Neural Radiance Fields from Multispectral and Panchromatic Acquisitions](https://mlsanity.com/paper/2409.15132/) (2024)
- [Enpowering Your Pansharpening Models with Generalizability: Unified Distribution is All You Need](https://mlsanity.com/paper/2510.22217/) (2025)
- [G-ZAP: A Generalizable Zero-Shot Framework for Arbitrary-Scale Pansharpening](https://mlsanity.com/paper/2603.14412/) (2026)
- [Two-Stage Random Alternation Framework for One-Shot Pansharpening](https://mlsanity.com/paper/2505.06576/) (2025)
- [THAT: Token-wise High-frequency Augmentation Transformer for Hyperspectral Pansharpening](https://mlsanity.com/paper/2508.08183/) (2025)

## Abstract (arXiv)

Pansharpening aims to generate a high spatial resolution multispectral image (HRMS) by fusing a low spatial resolution multispectral image (LRMS) and a panchromatic image (PAN). The most challenging issue for this task is that only the to-be-fused LRMS and PAN are available, and the existing deep learning-based methods are unsuitable since they rely on many training pairs. Traditional variational optimization (VO) based methods are well-suited for addressing such a problem. They focus on carefully designing explicit fusion rules as well as regularizations for an optimization problem, which are based on the researcher's discovery of the image relationships and image structures. Unlike previous VO-based methods, in this work, we explore such complex relationships by a parameterized term rather than a manually designed one. Specifically, we propose a zero-shot pansharpening method by introducing a neural network into the optimization objective. This network estimates a representation component of HRMS, which mainly describes the relationship between HRMS and PAN. In this way, the network achieves a similar goal to the so-called deep image prior because it implicitly regulates the relationship between the HRMS and PAN images through its inherent structure. We directly minimize this optimization objective via network parameters and the expected HRMS image through iterative updating. Extensive experiments on various benchmark datasets demonstrate that our proposed method can achieve better performance compared with other state-of-the-art methods. The codes are available at https://github.com/xyrui/PSDip.