# Language-Informed Hyperspectral Image Synthesis for Imbalanced-Small Sample Classification via Semi-Supervised Conditional Diffusion Model arXiv: https://arxiv.org/abs/2502.19700 ยท HTML: https://mlsanity.com/paper/2502.19700/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. - Builds on 34 of its 117 references that are in the map. - First posted on arXiv 27 Feb 2025. ## Placed in - Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/) ## Builds on (most cited first, 30 of 34) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019) - [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023) - [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021) - [SpectralGPT: Spectral Remote Sensing Foundation Model](https://mlsanity.com/paper/2311.07113/) (2023) - [RSGPT: A Remote Sensing Vision Language Model and Benchmark](https://mlsanity.com/paper/2307.15266/) (2023) - [SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery](https://mlsanity.com/paper/2312.10115/) (2023) - [SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers](https://mlsanity.com/paper/2107.02988/) (2021) - [DiffusionSat: A Generative Foundation Model for Satellite Imagery](https://mlsanity.com/paper/2312.03606/) (2023) - [Multimodal Fusion Transformer for Remote Sensing Image Classification](https://mlsanity.com/paper/2203.16952/) (2022) - [CRS-Diff: Controllable Remote Sensing Image Generation with Diffusion Model](https://mlsanity.com/paper/2403.11614/) (2024) - [Class-Balancing Diffusion Models](https://mlsanity.com/paper/2305.00562/) (2023) - [SSMix: Saliency-Based Span Mixup for Text Classification](https://mlsanity.com/paper/2106.08062/) (2021) - [SpectralDiff: A Generative Framework for Hyperspectral Image Classification with Diffusion Models](https://mlsanity.com/paper/2304.05961/) (2023) - [Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion](https://mlsanity.com/paper/2401.10786/) (2024) - [SS-MAE: Spatial-Spectral Masked Auto-Encoder for Multi-Source Remote Sensing Image Classification](https://mlsanity.com/paper/2311.04442/) (2023) - [Language-aware Domain Generalization Network for Cross-Scene Hyperspectral Image Classification](https://mlsanity.com/paper/2209.02700/) (2022) - [DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Change Detection](https://mlsanity.com/paper/2206.11892/) (2022) - [Exploring Multi-Timestep Multi-Stage Diffusion Features for Hyperspectral Image Classification](https://mlsanity.com/paper/2306.08964/) (2023) - [Latent-based Diffusion Model for Long-tailed Recognition](https://mlsanity.com/paper/2404.04517/) (2024) - [DCN-T: Dual Context Network with Transformer for Hyperspectral Image Classification](https://mlsanity.com/paper/2304.09915/) (2023) - [Generative Adversarial Networks for Realistic Synthesis of Hyperspectral Samples](https://mlsanity.com/paper/1806.02583/) (2018) - [Stable Diffusion Dataset Generation for Downstream Classification Tasks](https://mlsanity.com/paper/2405.02698/) (2024) - [GMM-Based Synthetic Samples for Classification of Hyperspectral Images With Limited Training Data](https://mlsanity.com/paper/1712.04778/) (2017) ## Built on by (most cited first, 1 of 1) - [Diffusion Models for Hyperspectral Image Analysis: A Comprehensive Review](https://mlsanity.com/paper/2505.11158/) (2025) ## Abstract (arXiv) Data augmentation effectively addresses the imbalanced-small sample data (ISSD) problem in hyperspectral image classification (HSIC). While most methodologies extend features in the latent space, few leverage text-driven generation to create realistic and diverse samples. Recently, text-guided diffusion models have gained significant attention due to their ability to generate highly diverse and high-quality images based on text prompts in natural image synthesis. Motivated by this, this paper proposes Txt2HSI-LDM(VAE), a novel language-informed hyperspectral image synthesis method to address the ISSD in HSIC. The proposed approach uses a denoising diffusion model, which iteratively removes Gaussian noise to generate hyperspectral samples conditioned on textual descriptions. First, to address the high-dimensionality of hyperspectral data, a universal variational autoencoder (VAE) is designed to map the data into a low-dimensional latent space, which provides stable features and reduces the inference complexity of diffusion model. Second, a semi-supervised diffusion model is designed to fully take advantage of unlabeled data. Random polygon spatial clipping (RPSC) and uncertainty estimation of latent feature (LF-UE) are used to simulate the varying degrees of mixing. Third, the VAE decodes HSI from latent space generated by the diffusion model with the language conditions as input. In our experiments, we fully evaluate synthetic samples' effectiveness from statistical characteristics and data distribution in 2D-PCA space. Additionally, visual-linguistic cross-attention is visualized on the pixel level to prove that our proposed model can capture the spatial layout and geometry of the generated data. Experiments demonstrate that the performance of the proposed Txt2HSI-LDM(VAE) surpasses the classical backbone models, state-of-the-art CNNs, and semi-supervised methods.