# Boosting Hyperspectral Image Classification with Gate-Shift-Fuse Mechanisms in a Novel CNN-Transformer Approach

arXiv: https://arxiv.org/abs/2406.14120 · HTML: https://mlsanity.com/paper/2406.14120/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 63 references that are in the map.
- First posted on arXiv 20 Jun 2024.

## Builds on (most cited first, 8 of 8)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Deep Learning for Hyperspectral Image Classification: An Overview](https://mlsanity.com/paper/1910.12861/) (2019)
- [HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification](https://mlsanity.com/paper/1902.06701/) (2019)
- [Gate-Shift-Fuse for Video Action Recognition](https://mlsanity.com/paper/2203.08897/) (2022)
- [DualMamba: A Lightweight Spectral-Spatial Mamba-Convolution Network for Hyperspectral Image Classification](https://mlsanity.com/paper/2406.07050/) (2024)
- [Deep Learning Techniques for Hyperspectral Image Analysis in Agriculture: A Review](https://mlsanity.com/paper/2304.13880/) (2023)
- [Manifold Alignment with Label Information](https://mlsanity.com/paper/2210.12774/) (2022)
- [Load Profile Inpainting for Missing Load Data Restoration and Baseline Estimation](https://mlsanity.com/paper/2211.16332/) (2022)

## Abstract (arXiv)

During the process of classifying Hyperspectral Image (HSI), every pixel sample is categorized under a land-cover type. CNN-based techniques for HSI classification have notably advanced the field by their adept feature representation capabilities. However, acquiring deep features remains a challenge for these CNN-based methods. In contrast, transformer models are adept at extracting high-level semantic features, offering a complementary strength. This paper's main contribution is the introduction of an HSI classification model that includes two convolutional blocks, a Gate-Shift-Fuse (GSF) block and a transformer block. This model leverages the strengths of CNNs in local feature extraction and transformers in long-range context modelling. The GSF block is designed to strengthen the extraction of local and global spatial-spectral features. An effective attention mechanism module is also proposed to enhance the extraction of information from HSI cubes. The proposed method is evaluated on four well-known datasets (the Indian Pines, Pavia University, WHU-WHU-Hi-LongKou and WHU-Hi-HanChuan), demonstrating that the proposed framework achieves superior results compared to other models.