mlsanityData to 14 Oct 2026

Clustered Federated Learning with Hierarchical Knowledge Distillation

arXiv 2512.10443 ↗First posted 11 Dec 2025cs.DC cs.AI cs.LGcited by 1

On the map: Knowledge distillation → Federated, robust and private distillation

Abstract (arXiv)

Clustered Federated Learning (CFL) has emerged as a powerful approach for addressing data heterogeneity and ensuring privacy in large distributed IoT environments. By clustering clients and training cluster-specific models, CFL enables personalized models tailored to groups of heterogeneous clients. However, conventional CFL approaches suffer from fragmented learning for training independent global models for each cluster and fail to take advantage of collective cluster insights. This paper advocates a shift to hierarchical CFL, allowing bi-level aggregation to train cluster-specific models at the edge and a unified global model at the cloud. This shift improves training efficiency yet might introduce communication challenges. To this end, we propose CFLHKD, a novel personalization scheme for integrating hierarchical cluster knowledge into CFL. Built upon multi-teacher knowledge distillation, CFLHKD enables inter-cluster knowledge sharing while preserving cluster-specific personalization. CFLHKD adopts a bi-level aggregation to bridge the gap between local and global learning. Extensive evaluations of standard benchmark datasets demonstrate that CFLHKD outperforms representative baselines in cluster-specific and global model accuracy and achieves a performance improvement of 3.32-7.57%.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Federated, robust and private distillation, one of 12 directions in Knowledge distillation. Builds on 14 of its 36 references that are in the map. First posted on arXiv 11 Dec 2025.

Builds on

14 of 36 references.

References · 14

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  6. 2018LEAF: A Benchmark for Federated Settingscited by 792
  7. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  8. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
  9. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  10. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  11. 2019Hierarchical Federated Learning Across Heterogeneous Cellular Networkscited by 98
  12. 2020FedGroup: Efficient Clustered Federated Learning via Decomposed Data-Driven Measurecited by 46
2 more
  1. 2022On the Convergence of Clustered Federated Learningcited by 31
  2. 2020ABBA: Adaptive Brownian bridge-based symbolic aggregation of time seriescited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Fed-BAC: Federated Bandit-Guided Additive Clustering in Hierarchical Federated Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Label driven Knowledge Distillation for Federated Learning with non-IID Data
  2. 2024Towards Diverse Device Heterogeneous Federated Learning via Task Arithmetic Knowledge Integration
  3. 2022HierarchyFL: Heterogeneous Federated Learning via Hierarchical Self-Distillation
  4. 2023Knowledge Distillation in Federated Edge Learning: A Survey
  5. 2024Towards Personalized Federated Learning via Comprehensive Knowledge Distillation
  6. 2022Enhancing Heterogeneous Federated Learning with Knowledge Extraction and Multi-Model Fusion
  7. 2024FedSiKD: Clients Similarity and Knowledge Distillation: Addressing Non-i.i.d. and Constraints in Federated Learning
  8. 2024Federated Knowledge Transfer Fine-tuning Large Server Model with Resource-Constrained IoT Clients