# Semantic Knowledge Distillation for Onboard Satellite Earth Observation Image Classification

arXiv: https://arxiv.org/abs/2411.00209 · HTML: https://mlsanity.com/paper/2411.00209/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 18 of 18)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Knowledge Distillation from A Stronger Teacher](https://mlsanity.com/paper/2205.10536/) (2022)
- [Does Knowledge Distillation Really Work?](https://mlsanity.com/paper/2106.05945/) (2021)
- [EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention](https://mlsanity.com/paper/2305.07027/) (2023)
- [Separable Self-attention for Mobile Vision Transformers](https://mlsanity.com/paper/2206.02680/) (2022)
- [Understanding Batch Normalization](https://mlsanity.com/paper/1806.02375/) (2018)
- [A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking](https://mlsanity.com/paper/2309.02031/) (2023)
- [Revisiting Knowledge Distillation: An Inheritance and Exploration Framework](https://mlsanity.com/paper/2107.00181/) (2021)
- [A Survey on Non-Geostationary Satellite Systems: The Communication Perspective](https://mlsanity.com/paper/2107.05312/) (2021)
- [Onboard Satellite Image Classification for Earth Observation: A Comparative Study of ViT Models](https://mlsanity.com/paper/2409.03901/) (2024)
- [Artificial Intelligence for Satellite Communication and Non-Terrestrial Networks: A Survey](https://mlsanity.com/paper/2304.13008/) (2023)
- [Edge AI Empowered Physical Layer Security for 6G NTN: Potential Threats and Future Opportunities](https://mlsanity.com/paper/2401.01005/) (2023)
- [On-Air Deep Learning Integrated Semantic Inference Models for Enhanced Earth Observation Satellite Networks](https://mlsanity.com/paper/2409.15246/) (2024)
- [Connecting Space Missions Through NGSO Constellations: Feasibility Study](https://mlsanity.com/paper/2309.16589/) (2023)
- [LEO-to-User Assignment and Resource Allocation for Uplink Transmit Power Minimization](https://mlsanity.com/paper/2302.01437/) (2023)
- [A Novel Transformer-Based Self-Supervised Learning Method to Enhance Photoplethysmogram Signal Artifact Detection](https://mlsanity.com/paper/2401.01013/) (2024)
- [Seamless 5G Automotive Connectivity with Integrated Satellite Terrestrial Networks in C-Band](https://mlsanity.com/paper/2408.15394/) (2024)

## Built on by (most cited first, 2 of 2)

- [A Semantic-Loss Function Modeling Framework With Task-Oriented Machine Learning Perspectives](https://mlsanity.com/paper/2503.09903/) (2025)
- [Label Less, Learn More: Resource-Efficient Active Semi-Supervised Learning for Onboard Satellite Image Annotation](https://mlsanity.com/paper/2609.37481/) (2026)

## Abstract (arXiv)

This study presents an innovative dynamic weighting knowledge distillation (KD) framework tailored for efficient Earth observation (EO) image classification (IC) in resource-constrained settings. Utilizing EfficientViT and MobileViT as teacher models, this framework enables lightweight student models, particularly ResNet8 and ResNet16, to surpass 90% in accuracy, precision, and recall, adhering to the stringent confidence thresholds necessary for reliable classification tasks. Unlike conventional KD methods that rely on static weight distribution, our adaptive weighting mechanism responds to each teacher model's confidence, allowing student models to prioritize more credible sources of knowledge dynamically. Remarkably, ResNet8 delivers substantial efficiency gains, achieving a 97.5% reduction in parameters, a 96.7% decrease in FLOPs, an 86.2% cut in power consumption, and a 63.5% increase in inference speed over MobileViT. This significant optimization of complexity and resource demands establishes ResNet8 as an optimal candidate for EO tasks, combining robust performance with feasibility in deployment. The confidence-based, adaptable KD approach underscores the potential of dynamic distillation strategies to yield high-performing, resource-efficient models tailored for satellite-based EO applications. The reproducible code is accessible on our GitHub repository.