mlsanityData to 14 Oct 2026

FedCC: Towards Addressing Label Distribution Skews in Distillation-Based Federated Learning

arXiv 2608.23031 ↗First posted 24 Aug 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) enables distributed clients to collaboratively train models without sharing raw data, making it promising for leveraging massive devices in communication networks. In distillation-based FL, each client applies its local model on an unlabeled public dataset, and shares only prediction results with the server. While heterogeneous local data introduces label distribution skew, thus biasing client models toward majority classes and leading to potentially inaccurate predictions. The lack of ground-truth labels in the public dataset hampers the server's ability to calibrate predictions, which ultimately degrades overall performance. To address this, we propose FedCC, a simple and effective algorithm for mitigating client misclassification. Instead of being forced to classify and risking error propagation, clients are allowed to tag ambiguous samples as 'unknown'. This additional class, together with calibrated pseudo-labels on the public data, balances confidence in majority classes against uncertainty in under-represented ones. Extensive experiments demonstrate that FedCC significantly outperforms existing methods, especially under severe label skew. In the extreme scenario where each client holds samples from only one of ten classes, FedCC achieves 67.3% accuracy, while baselines collapse to near-random results.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 32 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

23 of 32 references.

References · 23

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  6. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  7. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  8. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  9. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
  10. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  11. 2025PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selectioncited by 61
  12. 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 51
11 more
  1. 2023LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation -- Extended Versioncited by 50
  2. 2024Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boostingcited by 27
  3. 2023Exploiting Label Skews in Federated Learning with Model Concatenationcited by 14
  4. 2024Exploring Vacant Classes in Label-Skewed Federated Learningcited by 10
  5. 2024CDMAD: Class-Distribution-Mismatch-Aware Debiasing for Class-Imbalanced Semi-Supervised Learningcited by 10
  6. 2021Not All Knowledge Is Created Equal: Mutual Distillation of Confident Knowledgecited by 9
  7. 2024Federated Learning with Label-Masking Distillationcited by 7
  8. 2023Advancing Federated Learning in 6G: A Trusted Architecture with Graph-based Analysiscited by 6
  9. 2023Little is Enough: Boosting Privacy by Sharing Only Hard Labels in Federated Semi-Supervised Learningcited by 5
  10. 2024Federated Learning under Partially Class-Disjoint Data via Manifold Reshapingcited by 4
  11. 2026Select to Think: Unlocking SLM Potential with Local Sufficiencycited by 3

Built on by

Not cited yet.