# LD-GAN: Low-Dimensional Generative Adversarial Network for Spectral Image Generation with Variance Regularization

arXiv: https://arxiv.org/abs/2305.00132 · HTML: https://mlsanity.com/paper/2305.00132/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 49 references that are in the map.
- First posted on arXiv 29 Apr 2023.

## Builds on (most cited first, 18 of 18)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [Data Augmentation Generative Adversarial Networks](https://mlsanity.com/paper/1711.04340/) (2017)
- [Projected GANs Converge Faster](https://mlsanity.com/paper/2111.01007/) (2021)
- [Generalization and Equilibrium in Generative Adversarial Nets (GANs)](https://mlsanity.com/paper/1703.00573/) (2017)
- [How Well Generative Adversarial Networks Learn Distributions](https://mlsanity.com/paper/1811.03179/) (2018)
- [An error analysis of generative adversarial networks for learning distributions](https://mlsanity.com/paper/2105.13010/) (2021)
- [Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net](https://mlsanity.com/paper/1901.03281/) (2019)
- [Advances in Hyperspectral Image Classification: Earth monitoring with statistical learning methods](https://mlsanity.com/paper/1310.5107/) (2013)
- [Accurate Spectral Super-resolution from Single RGB Image Using Multi-scale CNN](https://mlsanity.com/paper/1806.03575/) (2018)
- [JR2net: A Joint Non-Linear Representation and Recovery Network for Compressive Spectral Imaging](https://mlsanity.com/paper/2205.07770/) (2022)
- [Computational Spectral Imaging: A Contemporary Overview](https://mlsanity.com/paper/2303.04848/) (2023)
- [Sparse Coding with Multi-Layer Decoders using Variance Regularization](https://mlsanity.com/paper/2112.09214/) (2021)
- [Deep Coding Patterns Design for Compressive Near-Infrared Spectral Classification](https://mlsanity.com/paper/2205.14069/) (2022)

## Built on by (most cited first, 2 of 2)

- [Statistical Guarantees of Group-Invariant GANs](https://mlsanity.com/paper/2305.13517/) (2023)
- [PlantPlotGAN: A Physics-Informed Generative Adversarial Network for Plant Disease Prediction](https://mlsanity.com/paper/2310.18268/) (2023)

## Abstract (arXiv)

Deep learning methods are state-of-the-art for spectral image (SI) computational tasks. However, these methods are constrained in their performance since available datasets are limited due to the highly expensive and long acquisition time. Usually, data augmentation techniques are employed to mitigate the lack of data. Surpassing classical augmentation methods, such as geometric transformations, GANs enable diverse augmentation by learning and sampling from the data distribution. Nevertheless, GAN-based SI generation is challenging since the high-dimensionality nature of this kind of data hinders the convergence of the GAN training yielding to suboptimal generation. To surmount this limitation, we propose low-dimensional GAN (LD-GAN), where we train the GAN employing a low-dimensional representation of the {dataset} with the latent space of a pretrained autoencoder network. Thus, we generate new low-dimensional samples which are then mapped to the SI dimension with the pretrained decoder network. Besides, we propose a statistical regularization to control the low-dimensional representation variance for the autoencoder training and to achieve high diversity of samples generated with the GAN. We validate our method LD-GAN as data augmentation strategy for compressive spectral imaging, SI super-resolution, and RBG to spectral tasks with improvements varying from 0.5 to 1 [dB] in each task respectively. We perform comparisons against the non-data augmentation training, traditional DA, and with the same GAN adjusted and trained to generate the full-sized SIs. The code of this paper can be found in https://github.com/romanjacome99/LD_GAN.git