# Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets

arXiv: https://arxiv.org/abs/2101.07589 · HTML: https://mlsanity.com/paper/2101.07589/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 64 references that are in the map.
- First posted on arXiv 19 Jan 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [Robust Learning Through Cross-Task Consistency](https://mlsanity.com/paper/2006.04096/) (2020)
- [Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2003.07018/) (2020)
- [Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Analysis and a New Strategy](https://mlsanity.com/paper/2004.00448/) (2020)
- [Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimation](https://mlsanity.com/paper/2012.10782/) (2020)
- [Hyperspectral pansharpening: a review](https://mlsanity.com/paper/1504.04531/) (2015)
- [Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery](https://mlsanity.com/paper/2005.08752/) (2020)
- [Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net](https://mlsanity.com/paper/1901.03281/) (2019)
- [NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image](https://mlsanity.com/paper/2005.03412/) (2020)
- [Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/1804.05042/) (2018)
- [Auxiliary Task Reweighting for Minimum-data Learning](https://mlsanity.com/paper/2010.08244/) (2020)
- [Learned Spectral Super-Resolution](https://mlsanity.com/paper/1703.09470/) (2017)

## Abstract (arXiv)

This work studies Hyperspectral image (HSI) super-resolution (SR). HSI SR is characterized by high-dimensional data and a limited amount of training examples. This exacerbates the undesirable behaviors of neural networks such as memorization and sensitivity to out-of-distribution samples. This work addresses these issues with three contributions. First, we observe that HSI SR and RGB image SR are correlated and develop a novel multi-tasking network to train them jointly so that the auxiliary task RGB image SR can provide additional supervision. Second, we propose a simple, yet effective data augmentation routine, termed Spectral Mixup, to construct effective virtual training samples to enlarge the training set. Finally, we extend the network to a semi-supervised setting so that it can learn from datasets containing only low-resolution HSIs. With these contributions, our method is able to learn from heterogeneous datasets and lift the requirement for having a large amount of HD HSI training samples. Extensive experiments on four standard datasets show that our method outperforms existing methods significantly and underpin the relevance of our contributions. Code has been made available at https://github.com/kli8996/HSISR.