mlsanityData to 14 Oct 2026

Geometric Knowledge-Assisted Federated Dual Knowledge Distillation Approach Towards Remote Sensing Satellite Imagery

arXiv 2603.07774 ↗First posted 8 Mar 2026cs.CVcited by 0

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

Federated learning (FL) has recently become a promising solution for analyzing remote sensing satellite imagery (RSSI). However, the large scale and inherent data heterogeneity of images collected from multiple satellites, where the local data distribution of each satellite differs from the global one, present significant challenges to effective model training. To address this issue, we propose a Geometric Knowledge-Guided Federated Dual Knowledge Distillation (GK-FedDKD) framework for RSSI analysis. In our approach, each local client first distills a teacher encoder (TE) from multiple student encoders (SEs) trained with unlabeled augmented data. The TE is then connected with a shared classifier to form a teacher network (TN) that supervises the training of a new student network (SN). The intermediate representations of the TN are used to compute local covariance matrices, which are aggregated at the server to generate global geometric knowledge (GGK). This GGK is subsequently employed for local embedding augmentation to further guide SN training. We also design a novel loss function and a multi-prototype generation pipeline to stabilize the training process. Evaluation over multiple datasets showcases that the proposed GK-FedDKD approach is superior to the considered state-of-the-art baselines, e.g., the proposed approach with the Swin-T backbone surpasses previous SOTA approaches by an average 68.89% on the EuroSAT dataset.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 32 of its 61 references that are in the map. First posted on arXiv 8 Mar 2026.

Builds on

32 of 61 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  7. 2021Model-Contrastive Federated Learningcited by 766
  8. 2019Federated Learning with Personalization Layerscited by 541
  9. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 500
  10. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  11. 2019Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocationcited by 94
  12. 2021Self-Guided and Cross-Guided Learning for Few-Shot Segmentationcited by 92
20 more
  1. 2014Convolutional Neural Networks at Constrained Time Costcited by 81
  2. 2023FedExP: Speeding Up Federated Averaging via Extrapolationcited by 69
  3. 2023FedDisco: Federated Learning with Discrepancy-Aware Collaborationcited by 67
  4. 2020FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tierscited by 64
  5. 2022Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learningcited by 52
  6. 2024FedGTA: Topology-aware Averaging for Federated Graph Learningcited by 44
  7. 2021Node Selection Toward Faster Convergence for Federated Learning on Non-IID Datacited by 39
  8. 2015DeepSat - A Learning framework for Satellite Imagerycited by 38
  9. 2024Relaxed Contrastive Learning for Federated Learningcited by 32
  10. 2023DiffCR: A Fast Conditional Diffusion Framework for Cloud Removal from Optical Satellite Imagescited by 27
  11. 2024FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuningcited by 27
  12. 2023A Simple Data Augmentation for Feature Distribution Skewed Federated Learningcited by 17
  13. 2024An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learningcited by 17
  14. 2017Max-Pooling Loss Training of Long Short-Term Memory Networks for Small-Footprint Keyword Spottingcited by 15
  15. 2025LCFed: An Efficient Clustered Federated Learning Framework for Heterogeneous Datacited by 15
  16. 2025Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learningcited by 12
  17. 2025Effective Cloud Removal for Remote Sensing Images by an Improved Mean-Reverting Denoising Model with Elucidated Design Spacecited by 12
  18. 2024Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Datacited by 11
  19. 2024Towards Satellite Non-IID Imagery: A Spectral Clustering-Assisted Federated Learning Approachcited by 4
  20. 2024Cyber Attacks Prevention Towards Prosumer-based EV Charging Stations: An Edge-assisted Federated Prototype Knowledge Distillation Approachcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022FedUKD: Federated UNet Model with Knowledge Distillation for Land Use Classification from Satellite and Street Views
  2. 2024Applications of Knowledge Distillation in Remote Sensing: A Survey
  3. 2025FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Models
  4. 2024Semantic Knowledge Distillation for Onboard Satellite Earth Observation Image Classification
  5. 2023Remote Sensing Object Counting with Online Knowledge Learning
  6. 2025Remote Sensing Image Classification with Decoupled Knowledge Distillation
  7. 2026GeoSANE: Learning Geospatial Representations from Models, Not Data
  8. 2025A Semi-Supervised Federated Learning Framework with Hierarchical Clustering Aggregation for Heterogeneous Satellite Networks