# 3D-RCNet: Learning from Transformer to Build a 3D Relational ConvNet for Hyperspectral Image Classification

arXiv: https://arxiv.org/abs/2408.13728 · HTML: https://mlsanity.com/paper/2408.13728/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 34 references that are in the map.
- First posted on arXiv 25 Aug 2024.

## Builds on (most cited first, 10 of 10)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer](https://mlsanity.com/paper/2110.02178/) (2021)
- [LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference](https://mlsanity.com/paper/2104.01136/) (2021)
- [SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers](https://mlsanity.com/paper/2107.02988/) (2021)
- [Visformer: The Vision-friendly Transformer](https://mlsanity.com/paper/2104.12533/) (2021)
- [HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification](https://mlsanity.com/paper/1902.06701/) (2019)
- [ParC-Net: Position Aware Circular Convolution with Merits from ConvNets and Transformer](https://mlsanity.com/paper/2203.03952/) (2022)
- [Hyperspectral Classification Based on Lightweight 3-D-CNN With Transfer Learning](https://mlsanity.com/paper/2012.03439/) (2020)
- [Fcaformer: Forward Cross Attention in Hybrid Vision Transformer](https://mlsanity.com/paper/2211.07198/) (2022)
- [IncepFormer: Efficient Inception Transformer with Pyramid Pooling for Semantic Segmentation](https://mlsanity.com/paper/2212.03035/) (2022)

## Built on by (most cited first, 1 of 1)

- [HyperCap: Hyperspectral Land Cover Captioning Dataset for Vision Language Models](https://mlsanity.com/paper/2505.12217/) (2025)

## Abstract (arXiv)

Recently, the Vision Transformer (ViT) model has replaced the classical Convolutional Neural Network (ConvNet) in various computer vision tasks due to its superior performance. Even in hyperspectral image (HSI) classification field, ViT-based methods also show promising potential. Nevertheless, ViT encounters notable difficulties in processing HSI data. Its self-attention mechanism, which exhibits quadratic complexity, escalates computational costs. Additionally, ViT's substantial demand for training samples does not align with the practical constraints posed by the expensive labeling of HSI data. To overcome these challenges, we propose a 3D relational ConvNet named 3D-RCNet, which inherits both strengths of ConvNet and ViT, resulting in high performance in HSI classification. We embed the self-attention mechanism of Transformer into the convolutional operation of ConvNet to design 3D relational convolutional operation and use it to build the final 3D-RCNet. The proposed 3D-RCNet maintains the high computational efficiency of ConvNet while enjoying the flexibility of ViT. Additionally, the proposed 3D relational convolutional operation is a plug-and-play operation, which can be inserted into previous ConvNet-based HSI classification methods seamlessly. Empirical evaluations on three representative benchmark HSI datasets show that the proposed model outperforms previous ConvNet-based and ViT-based HSI approaches.