# VP-Hype: A Hybrid Mamba-Transformer Framework with Visual-Textual Prompting for Hyperspectral Image Classification

arXiv: https://arxiv.org/abs/2603.01174 · HTML: https://mlsanity.com/paper/2603.01174/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Hyperspectral image analysis with Mamba, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 1 Mar 2026.

## Placed in

- State-space sequence models (S4, Mamba) / [Hyperspectral image analysis with Mamba](https://mlsanity.com/ssm/hyperspectral-image-analysis-with-mamba/)

## Builds on (most cited first, 14 of 14)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [Rethinking Spatial Dimensions of Vision Transformers](https://mlsanity.com/paper/2103.16302/) (2021)
- [InstructIR: High-Quality Image Restoration Following Human Instructions](https://mlsanity.com/paper/2401.16468/) (2024)
- [PromptIR: Prompting for All-in-One Blind Image Restoration](https://mlsanity.com/paper/2306.13090/) (2023)
- [HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification](https://mlsanity.com/paper/1902.06701/) (2019)
- [Prompt-In-Prompt Learning for Universal Image Restoration](https://mlsanity.com/paper/2312.05038/) (2023)
- [PromptHSI: Universal Hyperspectral Image Restoration with Vision-Language Modulated Frequency Adaptation](https://mlsanity.com/paper/2411.15922/) (2024)
- [MP-HSIR: A Multi-Prompt Framework for Universal Hyperspectral Image Restoration](https://mlsanity.com/paper/2503.09131/) (2025)
- [LoLA-SpecViT: Local Attention SwiGLU Vision Transformer with LoRA for Hyperspectral Imaging](https://mlsanity.com/paper/2506.17759/) (2025)

## Abstract (arXiv)

Accurate classification of hyperspectral imagery (HSI) is often frustrated by the tension between high-dimensional spectral data and the extreme scarcity of labeled training samples. While hierarchical models like LoLA-SpecViT have demonstrated the power of local windowed attention and parameter-efficient fine-tuning, the quadratic complexity of standard Transformers remains a barrier to scaling. We introduce VP-Hype, a framework that rethinks HSI classification by unifying the linear-time efficiency of State-Space Models (SSMs) with the relational modeling of Transformers in a novel hybrid architecture. Building on a robust 3D-CNN spectral front-end, VP-Hype replaces conventional attention blocks with a Hybrid Mamba-Transformer backbone to capture long-range dependencies with significantly reduced computational overhead. Furthermore, we address the label-scarcity problem by integrating dual-modal Visual and Textual Prompts that provide context-aware guidance for the feature extraction process. Our experimental evaluation demonstrates that VP-Hype establishes a new state of the art in low-data regimes. Specifically, with a training sample distribution of only 2%, the model achieves Overall Accuracy (OA) of 99.69% on the Salinas dataset and 99.45% on the Longkou dataset. These results suggest that the convergence of hybrid sequence modeling and multi-modal prompting provides a robust path forward for high-performance, sample-efficient remote sensing.