# SPECIAL: Zero-shot Hyperspectral Image Classification With CLIP

arXiv: https://arxiv.org/abs/2501.16222 · HTML: https://mlsanity.com/paper/2501.16222/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 55 references that are in the map.
- First posted on arXiv 27 Jan 2025.

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Extract Free Dense Labels from CLIP](https://mlsanity.com/paper/2112.01071/) (2021)
- [Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP](https://mlsanity.com/paper/2210.04150/) (2022)
- [ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2212.03588/) (2022)
- [SCLIP: Rethinking Self-Attention for Dense Vision-Language Inference](https://mlsanity.com/paper/2312.01597/) (2023)
- [CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2212.09506/) (2022)
- [ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inference](https://mlsanity.com/paper/2407.12442/) (2024)
- [Grounding Everything: Emerging Localization Properties in Vision-Language Transformers](https://mlsanity.com/paper/2312.00878/) (2023)
- [Going Deeper with Contextual CNN for Hyperspectral Image Classification](https://mlsanity.com/paper/1604.03519/) (2016)
- [MambaHSI: Spatial-Spectral Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2501.04944/) (2025)
- [SpectralMamba: Efficient Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2404.08489/) (2024)
- [SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2410.01768/) (2024)
- [IGroupSS-Mamba: Interval Group Spatial-Spectral Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2410.05100/) (2024)
- [Mask Approximation Net: A Novel Diffusion Model Approach for Remote Sensing Change Captioning](https://mlsanity.com/paper/2412.19179/) (2024)
- [AeroRIT: A New Scene for Hyperspectral Image Analysis](https://mlsanity.com/paper/1912.08178/) (2019)
- [Exploring Cross-Domain Pretrained Model for Hyperspectral Image Classification](https://mlsanity.com/paper/2204.03144/) (2022)
- [DiffCLIP: Few-shot Language-driven Multimodal Classifier](https://mlsanity.com/paper/2412.07119/) (2024)

## Built on by (most cited first, 4 of 4)

- [Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Images](https://mlsanity.com/paper/2508.18067/) (2025)
- [FSG-Net: Frequency-Spatial Synergistic Gated Network for High-Resolution Remote Sensing Change Detection](https://mlsanity.com/paper/2509.06482/) (2025)
- [SoDa2: Single-Stage Open-Set Domain Adaptation via Decoupled Alignment for Cross-Scene Hyperspectral Image Classification](https://mlsanity.com/paper/2605.03371/) (2026)
- [HyperSAM: A Promptable Foundation Model for Hyperspectral Remote Sensing](https://mlsanity.com/paper/2609.37340/) (2026)

## Abstract (arXiv)

Hyperspectral image (HSI) classification aims to categorize each pixel in an HSI into a specific land cover class, which is crucial for applications such as remote sensing, environmental monitoring, and agriculture. Although deep learning-based HSI classification methods have achieved significant advancements, existing methods still rely on manually labeled data for training, which is both time-consuming and labor-intensive. To address this limitation, we introduce a novel zero-shot hyperspectral image classification framework based on CLIP (SPECIAL), aiming to eliminate the need for manual annotations. The SPECIAL framework consists of two main stages: (1) CLIP-based pseudo-label generation, and (2) noisy label learning. In the first stage, HSI is spectrally interpolated to produce RGB bands. These bands are subsequently classified using CLIP, resulting in noisy pseudo-labels that are accompanied by confidence scores. To improve the quality of these labels, we propose a scaling strategy that fuses predictions from multiple spatial scales. In the second stage, spectral information and a label refinement technique are incorporated to mitigate label noise and further enhance classification accuracy. Experimental results on three benchmark datasets demonstrate that our SPECIAL outperforms existing methods in zero-shot HSI classification, showing its potential for more practical applications. The code is available at https://github.com/LiPang/SPECIAL.