# Cosine-Normalized Attention for Hyperspectral Image Classification

arXiv: https://arxiv.org/abs/2604.01763 · HTML: https://mlsanity.com/paper/2604.01763/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 43 references that are in the map.
- First posted on arXiv 2 Apr 2026.

## Builds on (most cited first, 15 of 15)

- [An Attentive Survey of Attention Models](https://mlsanity.com/paper/1904.02874/) (2019)
- [Hyperspectral Image Classification-Traditional to Deep Models: A Survey for Future Prospects](https://mlsanity.com/paper/2101.06116/) (2021)
- [A Comprehensive Survey for Hyperspectral Image Classification: The Evolution from Conventional to Transformers and Mamba Models](https://mlsanity.com/paper/2404.14955/) (2024)
- [WaveMamba: Spatial-Spectral Wavelet Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2408.01231/) (2024)
- [GraphMamba: An Efficient Graph Structure Learning Vision Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2407.08255/) (2024)
- [Multi-head Spatial-Spectral Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2408.01224/) (2024)
- [Transformers Fusion across Disjoint Samples for Hyperspectral Image Classification](https://mlsanity.com/paper/2405.01095/) (2024)
- [Spatial and Spatial-Spectral Morphological Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2408.01372/) (2024)
- [Attention Mechanism Meets with Hybrid Dense Network for Hyperspectral Image Classification](https://mlsanity.com/paper/2201.01001/) (2022)
- [Hyperspectral Sensors and Autonomous Driving: Technologies, Limitations, and Opportunities](https://mlsanity.com/paper/2508.19905/) (2025)
- [EnergyFormer: Energy Attention with Fourier Embedding for Hyperspectral Image Classification](https://mlsanity.com/paper/2503.08239/) (2025)
- [Transformer-Driven Active Transfer Learning for Cross-Hyperspectral Image Classification](https://mlsanity.com/paper/2411.18115/) (2024)
- [DiffFormer: a Differential Spatial-Spectral Transformer for Hyperspectral Image Classification](https://mlsanity.com/paper/2412.17350/) (2024)
- [EAUWSeg: Eliminating annotation uncertainty in weakly-supervised medical image segmentation](https://mlsanity.com/paper/2501.01658/) (2025)
- [3D Fourier-based Global Feature Extraction for Hyperspectral Image Classification](https://mlsanity.com/paper/2603.16426/) (2026)

## Abstract (arXiv)

Transformer-based methods have improved hyperspectral image classification (HSIC) by modeling long-range spatial-spectral dependencies; however, their attention mechanisms typically rely on dot-product similarity, which mixes feature magnitude and orientation and may be suboptimal for hyperspectral data. This work revisits attention scoring from a geometric perspective and introduces a cosine-normalized attention formulation that aligns similarity computation with the angular structure of hyperspectral signatures. By projecting query and key embeddings onto a unit hypersphere and applying a squared cosine similarity, the proposed method emphasizes angular relationships while reducing sensitivity to magnitude variations. The formulation is integrated into a spatial-spectral Transformer and evaluated under extremely limited supervision. Experiments on three benchmark datasets demonstrate that the proposed approach consistently achieves higher performance, outperforming several recent Transformer- and Mamba-based models despite using a lightweight backbone. In addition, a controlled analysis of multiple attention score functions shows that cosine-based scoring provides a reliable inductive bias for hyperspectral representation learning.