mlsanityData to 7 Oct 2026

FedICT: Federated Multi-task Distillation for Multi-access Edge Computing

arXiv 2301.00389 ↗First posted 1 Jan 2023cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

The growing interest in intelligent services and privacy protection for mobile devices has given rise to the widespread application of federated learning in Multi-access Edge Computing (MEC). Diverse user behaviors call for personalized services with heterogeneous Machine Learning (ML) models on different devices. Federated Multi-task Learning (FMTL) is proposed to train related but personalized ML models for different devices, whereas previous works suffer from excessive communication overhead during training and neglect the model heterogeneity among devices in MEC. Introducing knowledge distillation into FMTL can simultaneously enable efficient communication and model heterogeneity among clients, whereas existing methods rely on a public dataset, which is impractical in reality. To tackle this dilemma, Federated MultI-task Distillation for Multi-access Edge CompuTing (FedICT) is proposed. FedICT direct local-global knowledge aloof during bi-directional distillation processes between clients and the server, aiming to enable multi-task clients while alleviating client drift derived from divergent optimization directions of client-side local models. Specifically, FedICT includes Federated Prior Knowledge Distillation (FPKD) and Local Knowledge Adjustment (LKA). FPKD is proposed to reinforce the clients' fitting of local data by introducing prior knowledge of local data distributions. Moreover, LKA is proposed to correct the distillation loss of the server, making the transferred local knowledge better match the generalized representation. Experiments on three datasets show that FedICT significantly outperforms all compared benchmarks in various data heterogeneous and model architecture settings, achieving improved accuracy with less than 1.2% training communication overhead compared with FedAvg and no more than 75% training communication round compared with FedGKT.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 45 of its 71 references that are in the map. First posted on arXiv 1 Jan 2023.

Builds on

45 of 71 references.

References · 45

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2020Adaptive Federated Optimizationcited by 1,036
  5. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  6. 2017Federated Multi-Task Learningcited by 669
  7. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  8. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  9. 2021Towards Personalized Federated Learningcited by 462
  10. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  11. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  12. 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 312
33 more
  1. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  2. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  3. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  4. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  5. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  6. 2021Federated Learning for Smart Healthcare: A Surveycited by 182
  7. 2018Large scale distributed neural network training through online distillationcited by 144
  8. 2020Survey of Personalization Techniques for Federated Learningcited by 144
  9. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  10. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  11. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  12. 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 91
  13. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  14. 2020Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computingcited by 52
  15. 2021FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusioncited by 49
  16. 2020Unsupervised Multi-Target Domain Adaptation Through Knowledge Distillationcited by 46
  17. 2021Local-Global Knowledge Distillation in Heterogeneous Federated Learning with Non-IID Datacited by 31
  18. 2021Towards Model Agnostic Federated Learning Using Knowledge Distillationcited by 31
  19. 2020Towards Federated Learning in UAV-Enabled Internet of Vehicles: A Multi-Dimensional Contract-Matching Approachcited by 29
  20. 2021A New Look and Convergence Rate of Federated Multi-Task Learning with Laplacian Regularizationcited by 27
  21. 2020Federated Edge Learning : Design Issues and Challengescited by 26
  22. 2021Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfercited by 23
  23. 2022Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated Distillationcited by 22
  24. 2021Federated Learning with Taskonomy for Non-IID Datacited by 21
  25. 2022FEDIC: Federated Learning on Non-IID and Long-Tailed Data via Calibrated Distillationcited by 19
  26. 2020Self-organizing Democratized Learning: Towards Large-scale Distributed Learning Systemscited by 14
  27. 2022FedDTG:Federated Data-Free Knowledge Distillation via Three-Player Generative Adversarial Networkscited by 12
  28. 2022Communication-Efficient Federated Distillation with Active Data Samplingcited by 10
  29. 2021Computational Intelligence and Deep Learning for Next-Generation Edge-Enabled Industrial IoTcited by 9
  30. 2022FedGradNorm: Personalized Federated Gradient-Normalized Multi-Task Learningcited by 9
  31. 2021Spirit Distillation: A Model Compression Method with Multi-domain Knowledge Transfercited by 7
  32. 2018Custom Dual Transportation Mode Detection by Smartphone Devices Exploiting Sensor Diversitycited by 5
  33. 2021Spirit Distillation: Precise Real-time Semantic Segmentation of Road Scenes with Insufficient Datacited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 7; 2025: 5; 2026: 1.

17 papers.

Sort:

  1. 2023FedBone: Towards Large-Scale Federated Multi-Task Learningcited by 14
  2. 2024Federated Distillation: A Surveycited by 14
  3. 2023Federated Learning on Virtual Heterogeneous Data with Local-global Distillationcited by 6
  4. 2025FNBench: Benchmarking Robust Federated Learning against Noisy Labelscited by 6
  5. 2023Knowledge Distillation in Federated Edge Learning: A Surveycited by 5
  6. 2024Federated Class-Incremental Learning with New-Class Augmented Self-Distillationcited by 4
  7. 2023Improving Communication Efficiency of Federated Distillation via Accumulating Local Updatescited by 1
  8. 2024Logits Poisoning Attack in Federated Distillationcited by 1
  9. 2024FedCache 2.0: Federated Edge Learning with Knowledge Caching and Dataset Distillationcited by 1
  10. 2024Peak-Controlled Logits Poisoning Attack in Federated Distillationcited by 1
  11. 2025Beyond Model Scale Limits: End-Edge-Cloud Federated Learning with Self-Rectified Knowledge Agglomerationcited by 1
  12. 2025FedX: Explanation-Guided Pruning for Communication-Efficient Federated Learning in Remote Sensingcited by 1
5 more
  1. 2024FedSiKD: Clients Similarity and Knowledge Distillation: Addressing Non-i.i.d. and Constraints in Federated Learningcited by 0
  2. 2024Privacy-Enhanced Training-as-a-Service for On-Device Intelligence: Concept, Architectural Scheme, and Open Problemscited by 0
  3. 2025HEART: Achieving Timely Multi-Model Training for Vehicle-Edge-Cloud-Integrated Hierarchical Federated Learningcited by 0
  4. 2025ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systemscited by 0
  5. 2026A Multi-Prototype-Guided Federated Knowledge Distillation Approach in AI-RAN Enabled Multi-Access Edge Computing Systemcited by 0