# Guided Deep Generative Model-based Spatial Regularization for Multiband Imaging Inverse Problems

arXiv: https://arxiv.org/abs/2306.17197 · HTML: https://mlsanity.com/paper/2306.17197/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 55 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 18 of 18)

- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Hyperspectral Image Restoration via Total Variation Regularized Low-rank Tensor Decomposition](https://mlsanity.com/paper/1707.02477/) (2017)
- [A convex formulation for hyperspectral image superresolution via subspace-based regularization](https://mlsanity.com/paper/1411.4005/) (2014)
- [Fast Hyperspectral Image Denoising and Inpainting Based on Low-Rank and Sparse Representations](https://mlsanity.com/paper/2103.06842/) (2021)
- [Deep Hyperspectral Prior: Denoising, Inpainting, Super-Resolution](https://mlsanity.com/paper/1902.00301/) (2019)
- [Weighted Low-rank Tensor Recovery for Hyperspectral Image Restoration](https://mlsanity.com/paper/1709.00192/) (2017)
- [Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation](https://mlsanity.com/paper/1502.03121/) (2015)
- [PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening](https://mlsanity.com/paper/1805.03371/) (2018)
- [Hyperspectral and Multispectral Image Fusion based on a Sparse Representation](https://mlsanity.com/paper/1409.5729/) (2014)
- [Deep Plug-and-Play Prior for Hyperspectral Image Restoration](https://mlsanity.com/paper/2209.08240/) (2022)
- [Guided Deep Decoder: Unsupervised Image Pair Fusion](https://mlsanity.com/paper/2007.11766/) (2020)
- [Hyperspectral Image Super-Resolution via Deep Prior Regularization with Parameter Estimation](https://mlsanity.com/paper/2009.04237/) (2020)
- [Hyperspectral and multispectral image fusion under spectrally varying spatial blurs -- Application to high dimensional infrared astronomical imaging](https://mlsanity.com/paper/1912.11868/) (2019)
- [LADMM-Net: An Unrolled Deep Network For Spectral Image Fusion From Compressive Data](https://mlsanity.com/paper/2103.00940/) (2021)
- [Reconstruction of partially sampled multi-band images - Application to STEM-EELS imaging](https://mlsanity.com/paper/1802.10066/) (2018)
- [Simulated JWST datasets for multispectral and hyperspectral image fusion](https://mlsanity.com/paper/2001.02618/) (2020)

## Abstract (arXiv)

When adopting a model-based formulation, solving inverse problems encountered in multiband imaging requires to define spatial and spectral regularizations. In most of the works of the literature, spectral information is extracted from the observations directly to derive data-driven spectral priors. Conversely, the choice of the spatial regularization often boils down to the use of conventional penalizations (e.g., total variation) promoting expected features of the reconstructed image (e.g., piecewise constant). In this work, we propose a generic framework able to capitalize on an auxiliary acquisition of high spatial resolution to derive tailored data-driven spatial regularizations. This approach leverages on the ability of deep learning to extract high level features. More precisely, the regularization is conceived as a deep generative network able to encode spatial semantic features contained in this auxiliary image of high spatial resolution. To illustrate the versatility of this approach, it is instantiated to conduct two particular tasks, namely multiband image fusion and multiband image inpainting. Experimental results obtained on these two tasks demonstrate the benefit of this class of informed regularizations when compared to more conventional ones.