mlsanityData to 14 Oct 2026

UNIDEAL: Curriculum Knowledge Distillation Federated Learning

arXiv 2309.08961 ↗First posted 16 Sep 2023cs.LG cs.CR cs.DCcited by 4

On the map: Knowledge distillation → Federated, robust and private distillation

Abstract (arXiv)

Federated Learning (FL) has emerged as a promising approach to enable collaborative learning among multiple clients while preserving data privacy. However, cross-domain FL tasks, where clients possess data from different domains or distributions, remain a challenging problem due to the inherent heterogeneity. In this paper, we present UNIDEAL, a novel FL algorithm specifically designed to tackle the challenges of cross-domain scenarios and heterogeneous model architectures. The proposed method introduces Adjustable Teacher-Student Mutual Evaluation Curriculum Learning, which significantly enhances the effectiveness of knowledge distillation in FL settings. We conduct extensive experiments on various datasets, comparing UNIDEAL with state-of-the-art baselines. Our results demonstrate that UNIDEAL achieves superior performance in terms of both model accuracy and communication efficiency. Additionally, we provide a convergence analysis of the algorithm, showing a convergence rate of O(1/T) under non-convex conditions.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Federated, robust and private distillation, one of 12 directions in Knowledge distillation. Builds on 23 of its 31 references that are in the map. First posted on arXiv 16 Sep 2023.

Builds on

23 of 31 references.

Closest ancestors · 3

Same direction, newest first

  1. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillation30 Aug
  2. 2021Local-Global Knowledge Distillation in Heterogeneous Federated Learning with Non-IID Data30 Jun
  3. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learning20 May

Other references · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2019Searching for MobileNetV3cited by 1,994
  8. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  9. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  10. 2021Towards Personalized Federated Learningcited by 464
  11. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  12. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
8 more
  1. 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 309
  2. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
  3. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
  4. 2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 122
  5. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  6. 2022CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learningcited by 34
  7. 2019Federated User Representation Learningcited by 32
  8. 2022Architecture Agnostic Federated Learning for Neural Networkscited by 30

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 1.

4 papers.

  1. 2023FedHCA2: Towards Hetero-Client Federated Multi-Task Learningcited by 17
  2. 2024Federated Distillation: A Surveycited by 15
  3. 2024Applications of Knowledge Distillation in Remote Sensing: A Surveycited by 8
  4. 2025Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environmentscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025SFedKD: Sequential Federated Learning with Discrepancy-Aware Multi-Teacher Knowledge Distillation
  2. 2023Selective Knowledge Sharing for Privacy-Preserving Federated Distillation without A Good Teacher
  3. 2026FedMTFI: Feature Importance Based Optimized Multi Teacher Knowledge Distillation in Heterogeneous Federated Learning Environment
  4. 2025Tackling Data Heterogeneity in Federated Learning through Knowledge Distillation with Inequitable Aggregation
  5. 2022Federated Unlearning with Knowledge Distillation
  6. 2024KnFu: Effective Knowledge Fusion
  7. 2026CRAD: Class-wise Reliability-Aware Distillation for Decentralized Heterogeneous Federated Learning
  8. 2022FedX: Unsupervised Federated Learning with Cross Knowledge Distillation