# Pixel-to-Abundance Translation: Conditional Generative Adversarial Networks Based on Patch Transformer for Hyperspectral Unmixing

arXiv: https://arxiv.org/abs/2312.13127 · HTML: https://mlsanity.com/paper/2312.13127/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 62 references that are in the map.
- First posted on arXiv 20 Dec 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth](https://mlsanity.com/paper/2103.03404/) (2021)
- [Generative Image Modeling using Style and Structure Adversarial Networks](https://mlsanity.com/paper/1603.05631/) (2016)
- [EndNet: Sparse AutoEncoder Network for Endmember Extraction and Hyperspectral Unmixing](https://mlsanity.com/paper/1708.01894/) (2017)
- [Hyperspectral Unmixing via Deep Autoencoder Networks for a Generalized Linear-Mixture/Nonlinear-Fluctuation Model](https://mlsanity.com/paper/1904.13017/) (2019)
- [Spectral Unmixing via Data-guided Sparsity](https://mlsanity.com/paper/1403.3155/) (2014)
- [Robust Collaborative Nonnegative Matrix Factorization For Hyperspectral Unmixing (R-CoNMF)](https://mlsanity.com/paper/1506.04870/) (2015)

## Built on by (most cited first, 3 of 3)

- [Transformer for Multitemporal Hyperspectral Image Unmixing](https://mlsanity.com/paper/2407.10427/) (2024)
- [An Elliptic Kernel Unsupervised Autoencoder-Graph Convolutional Network Ensemble Model for Hyperspectral Unmixing](https://mlsanity.com/paper/2406.06742/) (2024)
- [ReflectGAN: Modeling Vegetation Effects for Soil Carbon Estimation from Satellite Imagery](https://mlsanity.com/paper/2505.18546/) (2025)

## Abstract (arXiv)

Spectral unmixing is a significant challenge in hyperspectral image processing. Existing unmixing methods utilize prior knowledge about the abundance distribution to solve the regularization optimization problem, where the difficulty lies in choosing appropriate prior knowledge and solving the complex regularization optimization problem. To solve these problems, we propose a hyperspectral conditional generative adversarial network (HyperGAN) method as a generic unmixing framework, based on the following assumption: the unmixing process from pixel to abundance can be regarded as a transformation of two modalities with an internal specific relationship. The proposed HyperGAN is composed of a generator and discriminator, the former completes the modal conversion from mixed hyperspectral pixel patch to the abundance of corresponding endmember of the central pixel and the latter is used to distinguish whether the distribution and structure of generated abundance are the same as the true ones. We propose hyperspectral image (HSI) Patch Transformer as the main component of the generator, which utilize adaptive attention score to capture the internal pixels correlation of the HSI patch and leverage the spatial-spectral information in a fine-grained way to achieve optimization of the unmixing process. Experiments on synthetic data and real hyperspectral data achieve impressive results compared to state-of-the-art competitors.