Semantic Knowledge Distillation for Onboard Satellite Earth Observation Image Classification
On the map: Knowledge distillation → Detection and dense prediction distillation
Abstract (arXiv)
This study presents an innovative dynamic weighting knowledge distillation (KD) framework tailored for efficient Earth observation (EO) image classification (IC) in resource-constrained settings. Utilizing EfficientViT and MobileViT as teacher models, this framework enables lightweight student models, particularly ResNet8 and ResNet16, to surpass 90% in accuracy, precision, and recall, adhering to the stringent confidence thresholds necessary for reliable classification tasks. Unlike conventional KD methods that rely on static weight distribution, our adaptive weighting mechanism responds to each teacher model's confidence, allowing student models to prioritize more credible sources of knowledge dynamically. Remarkably, ResNet8 delivers substantial efficiency gains, achieving a 97.5% reduction in parameters, a 96.7% decrease in FLOPs, an 86.2% cut in power consumption, and a 63.5% increase in inference speed over MobileViT. This significant optimization of complexity and resource demands establishes ResNet8 as an optimal candidate for EO tasks, combining robust performance with feasibility in deployment. The confidence-based, adaptable KD approach underscores the potential of dynamic distillation strategies to yield high-performing, resource-efficient models tailored for satellite-based EO applications. The reproducible code is accessible on our GitHub repository.
Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 18 of its 33 references that are in the map. First posted on arXiv 31 Oct 2024.
Builds on
18 of 33 references.
References · 18
- 2015Distilling the Knowledge in a Neural Networkcited by 10,180
- 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,492
- 2022Knowledge Distillation from A Stronger Teachercited by 204
- 2021Does Knowledge Distillation Really Work?cited by 188
- 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
- 2022Separable Self-attention for Mobile Vision Transformerscited by 163
- 2018Understanding Batch Normalizationcited by 158
- 2023A survey on efficient vision transformers: algorithms, techniques, and performance benchmarkingcited by 41
- 2021Revisiting Knowledge Distillation: An Inheritance and Exploration Frameworkcited by 15
- 2021A Survey on Non-Geostationary Satellite Systems: The Communication Perspectivecited by 12
- 2024Onboard Satellite Image Classification for Earth Observation: A Comparative Study of ViT Modelscited by 8
- 2023Artificial Intelligence for Satellite Communication and Non-Terrestrial Networks: A Surveycited by 6
6 more
- 2023Edge AI Empowered Physical Layer Security for 6G NTN: Potential Threats and Future Opportunitiescited by 4
- 2024On-Air Deep Learning Integrated Semantic Inference Models for Enhanced Earth Observation Satellite Networkscited by 4
- 2023Connecting Space Missions Through NGSO Constellations: Feasibility Studycited by 2
- 2023LEO-to-User Assignment and Resource Allocation for Uplink Transmit Power Minimizationcited by 1
- 2024A Novel Transformer-Based Self-Supervised Learning Method to Enhance Photoplethysmogram Signal Artifact Detectioncited by 1
- 2024Seamless 5G Automotive Connectivity with Integrated Satellite Terrestrial Networks in C-Bandcited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.