mlsanityData to 14 Oct 2026

CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learning

arXiv 2204.03880 ↗First posted 8 Apr 2022cs.CV cs.LGcited by 34

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a distributed learning paradigm that enables multiple clients to collaboratively learn a shared global model. Despite the recent progress, it remains challenging to deal with heterogeneous data clients, as the discrepant data distributions usually prevent the global model from delivering good generalization ability on each participating client. In this paper, we propose CD^2-pFed, a novel Cyclic Distillation-guided Channel Decoupling framework, to personalize the global model in FL, under various settings of data heterogeneity. Different from previous works which establish layer-wise personalization to overcome the non-IID data across different clients, we make the first attempt at channel-wise assignment for model personalization, referred to as channel decoupling. To further facilitate the collaboration between private and shared weights, we propose a novel cyclic distillation scheme to impose a consistent regularization between the local and global model representations during the federation. Guided by the cyclical distillation, our channel decoupling framework can deliver more accurate and generalized results for different kinds of heterogeneity, such as feature skew, label distribution skew, and concept shift. Comprehensive experiments on four benchmarks, including natural image and medical image analysis tasks, demonstrate the consistent effectiveness of our method on both local and external validations.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 8 Apr 2022.

Builds on

36 of 49 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  7. 2019Advances and Open Problems in Federated Learningcited by 3,046
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  9. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  10. 2018Federated Learning with Non-IID Datacited by 1,108
  11. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  12. 2019Towards Federated Learning at Scale: System Designcited by 832
24 more
  1. 2021Model-Contrastive Federated Learningcited by 766
  2. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  3. 2017Federated Multi-Task Learningcited by 672
  4. 2015Cyclical Learning Rates for Training Neural Networkscited by 629
  5. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  6. 2019Federated Learning with Personalization Layerscited by 541
  7. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 472
  8. 2021Towards Personalized Federated Learningcited by 464
  9. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  10. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  11. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  12. 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 287
  13. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  14. 2019Universally Slimmable Networks and Improved Training Techniquescited by 221
  15. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  16. 2020Survey of Personalization Techniques for Federated Learningcited by 145
  17. 2020Regularizing Class-wise Predictions via Self-knowledge Distillationcited by 144
  18. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  19. 2020Knowledge Distillation Meets Self-Supervisioncited by 142
  20. 2020Provable Defense against Privacy Leakage in Federated Learning from Representation Perspectivecited by 94
  21. 2021Multi-institutional Collaborations for Improving Deep Learning-based Magnetic Resonance Image Reconstruction Using Federated Learningcited by 78
  22. 2021Collaborative Unsupervised Visual Representation Learning from Decentralized Datacited by 69
  23. 2020Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge based Frameworkcited by 59
  24. 2019Personalization of Deep Learningcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 17; 2025: 4; 2026: 1.

34 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  2. 2023Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generationcited by 54
  3. 2023Federated Domain Generalization: A Surveycited by 25
  4. 2024Federated Distillation: A Surveycited by 15
  5. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  6. 2023FedWon: Triumphing Multi-domain Federated Learning Without Normalizationcited by 11
  7. 2024FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Datacited by 10
  8. 2024FedUV: Uniformity and Variance for Heterogeneous Federated Learningcited by 9
  9. 2024Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutionscited by 9
  10. 2024Spectral Co-Distillation for Personalized Federated Learningcited by 8
  11. 2024A Systematic Review of Generalization Research in Medical Image Classificationcited by 8
  12. 2023Fed-CO2: Cooperation of Online and Offline Models for Severe Data Heterogeneity in Federated Learningcited by 7
22 more
  1. 2024Influence-Oriented Personalized Federated Learningcited by 7
  2. 2022Personalized Federated Learning with Communication Compressioncited by 6
  3. 2023Federated Learning of Shareable Bases for Personalization-Friendly Image Classificationcited by 6
  4. 2024Federated brain tumor segmentation: an extensive benchmarkcited by 5
  5. 2023A Client-server Deep Federated Learning for Cross-domain Surgical Image Segmentationcited by 4
  6. 2023UNIDEAL: Curriculum Knowledge Distillation Federated Learningcited by 4
  7. 2025Spatio-Temporal Representation Decoupling and Enhancement for Federated Instrument Segmentation in Surgical Videoscited by 4
  8. 2024Personalizing Federated Instrument Segmentation with Visual Trait Priors in Robotic Surgerycited by 3
  9. 2025Federated Client-tailored Adapter for Medical Image Segmentationcited by 3
  10. 2023Robust Knowledge Adaptation for Federated Unsupervised Person ReIDcited by 2
  11. 2024DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representationscited by 2
  12. 2024Privacy-Preserving Federated Learning with Consistency via Knowledge Distillation Using Conditional Generatorcited by 2
  13. 2024Asymmetrical Reciprocity-based Federated Learning for Resolving Disparities in Medical Diagnosiscited by 2
  14. 2025Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptationcited by 2
  15. 2024FedPartWhole: Federated domain generalization via consistent part-whole hierarchiescited by 1
  16. 2024Personalized Federated Learning on Heterogeneous and Long-Tailed Data via Expert Collaborative Learningcited by 1
  17. 2022One-Time Model Adaptation to Heterogeneous Clients: An Intra-Client and Inter-Image Attention Designcited by 0
  18. 2023Factor-Assisted Federated Learning for Personalized Optimization with Heterogeneous Datacited by 0
  19. 2024Tailored Federated Learning: Leveraging Direction Regulation & Knowledge Distillationcited by 0
  20. 2024Delayed Random Partial Gradient Averaging for Federated Learningcited by 0
  21. 2025UniVarFL: Uniformity and Variance Regularized Federated Learning for Heterogeneous Datacited by 0
  22. 2026SurgFed: Language-guided Multi-Task Federated Learning for Surgical Video Understandingcited by 0