mlsanityData to 14 Oct 2026

HSI Image Enhancement Classification Based on Knowledge Distillation: A Study on Forgetting

arXiv 2603.20292 ↗First posted 19 Mar 2026cs.CV cs.AI cs.LGcited by 0

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

In incremental classification tasks for hyperspectral images, catastrophic forgetting is an unavoidable challenge. While memory recall methods can mitigate this issue, they heavily rely on samples from old categories. This paper proposes a teacher-based knowledge retention method for incremental image classification. It alleviates model forgetting of old category samples by utilizing incremental category samples, without depending on old category samples. Additionally, this paper introduces a mask-based partial category knowledge distillation algorithm. By decoupling knowledge distillation, this approach filters out potentially misleading information that could misguide the student model, thereby enhancing overall accuracy. Comparative and ablation experiments demonstrate the proposed method's robust performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 29 of its 50 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

29 of 50 references.

References · 29

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2016Learning without Forgettingcited by 2,496
  4. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  5. 2020Knowledge Distillation: A Surveycited by 1,603
  6. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  7. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  8. 2019Relational Knowledge Distillationcited by 889
  9. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  10. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
  11. 2019Three scenarios for continual learningcited by 614
  12. 2018End-to-End Incremental Learningcited by 597
17 more
  1. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  2. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 449
  3. 2022Decoupled Knowledge Distillationcited by 388
  4. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  5. 2018Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neuronscited by 238
  6. 2022FOSTER: Feature Boosting and Compression for Class-Incremental Learningcited by 229
  7. 2018Knowledge Distillation by On-the-Fly Native Ensemblecited by 186
  8. 2019Class-incremental Learning via Deep Model Consolidationcited by 173
  9. 2018Large scale distributed neural network training through online distillationcited by 146
  10. 2017Data-Free Knowledge Distillation for Deep Neural Networkscited by 136
  11. 2020Data-Free Network Quantization With Adversarial Knowledge Distillationcited by 80
  12. 2023Class-Incremental Exemplar Compression for Class-Incremental Learningcited by 50
  13. 2023Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learningcited by 41
  14. 2019Feature Fusion for Online Mutual Knowledge Distillationcited by 40
  15. 2023Parameter-Level Soft-Masking for Continual Learningcited by 40
  16. 2022Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replaycited by 35
  17. 2020Online Ensemble Model Compression using Knowledge Distillationcited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SDHSI-Net: Learning Better Representations for Hyperspectral Images via Self-Distillation
  2. 2021Response-based Distillation for Incremental Object Detection
  3. 2022Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation
  4. 2024CSAKD: Knowledge Distillation with Cross Self-Attention for Hyperspectral and Multispectral Image Fusion
  5. 2025Remote Sensing Image Classification with Decoupled Knowledge Distillation
  6. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning
  7. 2024Bridge Past and Future: Overcoming Information Asymmetry in Incremental Object Detection
  8. 2024Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration