# Model Inspired Autoencoder for Unsupervised Hyperspectral Image Super-Resolution

arXiv: https://arxiv.org/abs/2110.11591 · HTML: https://mlsanity.com/paper/2110.11591/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 71 references that are in the map.
- First posted on arXiv 22 Oct 2021.

## Builds on (most cited first, 18 of 18)

- [Hyperspectral pansharpening: a review](https://mlsanity.com/paper/1504.04531/) (2015)
- [A Multi-Scale and Multi-Depth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening](https://mlsanity.com/paper/1712.09809/) (2017)
- [A convex formulation for hyperspectral image superresolution via subspace-based regularization](https://mlsanity.com/paper/1411.4005/) (2014)
- [Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery](https://mlsanity.com/paper/2005.08752/) (2020)
- [Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net](https://mlsanity.com/paper/1901.03281/) (2019)
- [Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution](https://mlsanity.com/paper/2007.05230/) (2020)
- [Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/1804.05042/) (2018)
- [Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation](https://mlsanity.com/paper/1502.03121/) (2015)
- [Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach](https://mlsanity.com/paper/1804.05307/) (2018)
- [Hyperspectral and Multispectral Image Fusion based on a Sparse Representation](https://mlsanity.com/paper/1409.5729/) (2014)
- [Recent Advances and New Guidelines on Hyperspectral and Multispectral Image Fusion](https://mlsanity.com/paper/2008.03426/) (2020)
- [Target-adaptive CNN-based pansharpening](https://mlsanity.com/paper/1709.06054/) (2017)
- [Guided Deep Decoder: Unsupervised Image Pair Fusion](https://mlsanity.com/paper/2007.11766/) (2020)
- [Coupled Convolutional Neural Network with Adaptive Response Function Learning for Unsupervised Hyperspectral Super-Resolution](https://mlsanity.com/paper/2007.14007/) (2020)
- [Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network](https://mlsanity.com/paper/1706.05249/) (2017)
- [Blind Image Fusion for Hyperspectral Imaging with the Directional Total Variation](https://mlsanity.com/paper/1710.05705/) (2017)
- [Spatial-Spectral Fusion by Combining Deep Learning and Variation Model](https://mlsanity.com/paper/1809.00764/) (2018)
- [Hyperspectral Super-Resolution via Global-Local Low-Rank Matrix Estimation](https://mlsanity.com/paper/1907.01149/) (2019)

## Built on by (most cited first, 10 of 10)

- [Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review](https://mlsanity.com/paper/2205.01380/) (2022)
- [Unsupervised Hyperspectral Pansharpening via Low-rank Diffusion Model](https://mlsanity.com/paper/2305.10925/) (2023)
- [A Spectral Diffusion Prior for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2311.08955/) (2023)
- [Unsupervised Hyperspectral and Multispectral Image Blind Fusion Based on Deep Tucker Decomposition Network with Spatial-Spectral Manifold Learning](https://mlsanity.com/paper/2409.09670/) (2024)
- [Cross-Spatial Pixel Integration and Cross-Stage Feature Fusion Based Transformer Network for Remote Sensing Image Super-Resolution](https://mlsanity.com/paper/2307.02974/) (2023)
- [Physics-Inspired Degradation Models for Hyperspectral Image Fusion](https://mlsanity.com/paper/2402.02411/) (2024)
- [SpectraLift: Physics-Guided Spectral-Inversion Network for Self-Supervised Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2507.13339/) (2025)
- [Physics-Informed Spectral Modeling for Hyperspectral Imaging](https://mlsanity.com/paper/2508.21618/) (2025)
- [SpectraMorph: Structured Latent Learning for Self-Supervised Hyperspectral Super-Resolution](https://mlsanity.com/paper/2510.20814/) (2025)
- [HyperBench: Standardizing and Scaling Synthetic Evaluation for Hyperspectral Super-Resolution](https://mlsanity.com/paper/2605.21671/) (2026)

## Abstract (arXiv)

This paper focuses on hyperspectral image (HSI) super-resolution that aims to fuse a low-spatial-resolution HSI and a high-spatial-resolution multispectral image to form a high-spatial-resolution HSI (HR-HSI). Existing deep learning-based approaches are mostly supervised that rely on a large number of labeled training samples, which is unrealistic. The commonly used model-based approaches are unsupervised and flexible but rely on hand-craft priors. Inspired by the specific properties of model, we make the first attempt to design a model inspired deep network for HSI super-resolution in an unsupervised manner. This approach consists of an implicit autoencoder network built on the target HR-HSI that treats each pixel as an individual sample. The nonnegative matrix factorization (NMF) of the target HR-HSI is integrated into the autoencoder network, where the two NMF parts, spectral and spatial matrices, are treated as decoder parameters and hidden outputs respectively. In the encoding stage, we present a pixel-wise fusion model to estimate hidden outputs directly, and then reformulate and unfold the model's algorithm to form the encoder network. With the specific architecture, the proposed network is similar to a manifold prior-based model, and can be trained patch by patch rather than the entire image. Moreover, we propose an additional unsupervised network to estimate the point spread function and spectral response function. Experimental results conducted on both synthetic and real datasets demonstrate the effectiveness of the proposed approach.