# SDHSI-Net: Learning Better Representations for Hyperspectral Images via Self-Distillation

arXiv: https://arxiv.org/abs/2601.07416 · HTML: https://mlsanity.com/paper/2601.07416/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 12 Jan 2026.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 8 of 8)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification](https://mlsanity.com/paper/1902.06701/) (2019)
- [Three dimensional Deep Learning approach for remote sensing image classification](https://mlsanity.com/paper/1806.05824/) (2018)
- [IGroupSS-Mamba: Interval Group Spatial-Spectral Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2410.05100/) (2024)
- [Spectral-Spatial Classification of Hyperspectral Image Using Autoencoders](https://mlsanity.com/paper/1511.02916/) (2015)
- [DiffSpectralNet : Unveiling the Potential of Diffusion Models for Hyperspectral Image Classification](https://mlsanity.com/paper/2312.12441/) (2023)

## Abstract (arXiv)

Hyperspectral image (HSI) classification presents unique challenges due to its high spectral dimensionality and limited labeled data. Traditional deep learning models often suffer from overfitting and high computational costs. Self-distillation (SD), a variant of knowledge distillation where a network learns from its own predictions, has recently emerged as a promising strategy to enhance model performance without requiring external teacher networks. In this work, we explore the application of SD to HSI by treating earlier outputs as soft targets, thereby enforcing consistency between intermediate and final predictions. This process improves intra-class compactness and inter-class separability in the learned feature space. Our approach is validated on two benchmark HSI datasets and demonstrates significant improvements in classification accuracy and robustness, highlighting the effectiveness of SD for spectral-spatial learning. Codes are available at https://github.com/Prachet-Dev-Singh/SDHSI.