mlsanityData to 14 Oct 2026

Exploring Vacant Classes in Label-Skewed Federated Learning

arXiv 2401.02329 ↗First posted 4 Jan 2024cs.LG cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Label skews, characterized by disparities in local label distribution across clients, pose a significant challenge in federated learning. As minority classes suffer from worse accuracy due to overfitting on local imbalanced data, prior methods often incorporate class-balanced learning techniques during local training. Although these methods improve the mean accuracy across all classes, we observe that vacant classes-referring to categories absent from a client's data distribution-remain poorly recognized. Besides, there is still a gap in the accuracy of local models on minority classes compared to the global model. This paper introduces FedVLS, a novel approach to label-skewed federated learning that integrates both vacant-class distillation and logit suppression simultaneously. Specifically, vacant-class distillation leverages knowledge distillation during local training on each client to retain essential information related to vacant classes from the global model. Moreover, logit suppression directly penalizes network logits for non-label classes, effectively addressing misclassifications in minority classes that may be biased toward majority classes. Extensive experiments validate the efficacy of FedVLS, demonstrating superior performance compared to previous state-of-the-art (SOTA) methods across diverse datasets with varying degrees of label skews. Our code is available at https://github.com/krumpguo/FedVLS.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 34 of its 54 references that are in the map. First posted on arXiv 4 Jan 2024.

Builds on

34 of 54 references.

References · 34

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  6. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  7. 2021Model-Contrastive Federated Learningcited by 766
  8. 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  10. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  11. 2020Long-tail learning via logit adjustmentcited by 618
  12. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
22 more
  1. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 564
  2. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  3. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  4. 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 309
  5. 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
  6. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
  7. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  8. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  9. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  10. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  11. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
  12. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  13. 2020Inverse Distance Aggregation for Federated Learning with Non-IID Datacited by 44
  14. 2022Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferringcited by 28
  15. 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 24
  16. 2023Fed-GraB: Federated Long-tailed Learning with Self-Adjusting Gradient Balancercited by 17
  17. 2023A Survey for Federated Learning Evaluations: Goals and Measurescited by 16
  18. 2023A Comparative Evaluation of FedAvg and Per-FedAvg Algorithms for Dirichlet Distributed Heterogeneous Datacited by 14
  19. 2023Exploiting Label Skews in Federated Learning with Model Concatenationcited by 14
  20. 2023A Survey on Class Imbalance in Federated Learningcited by 13
  21. 2023Federated Skewed Label Learning with Logits Fusioncited by 5
  22. 2023Federated Learning with Classifier Shift for Class Imbalancecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 3.

10 papers.

  1. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 26
  2. 2024Towards Eliminating Hard Label Constraints in Gradient Inversion Attackscited by 4
  3. 2024Dataset Distillation-based Hybrid Federated Learning on Non-IID Datacited by 3
  4. 2025Neural Collapse-Inspired Multi-Label Federated Learning under Label-Distribution Skewcited by 2
  5. 2025Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Modelscited by 1
  6. 2024FedDW: Distilling Weights through Consistency Optimization in Heterogeneous Federated Learningcited by 0
  7. 2025Cooperative Pseudo Labeling for Unsupervised Federated Classificationcited by 0
  8. 2026Self-Supervised Federated Learning under Data Heterogeneity for Label-Scarce Diatom Classificationcited by 0
  9. 2026WHERE to Generate Matters: Budget-Aware Synthetic Augmentation for Label Skewed Federated Learningcited by 0
  10. 2026FedCC: Towards Addressing Label Distribution Skews in Distillation-Based Federated Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Partial Knowledge Distillation for Alleviating the Inherent Inter-Class Discrepancy in Federated Learning
  2. 2026CALM: Class-wise Agreement and Label-gated Disagreement Modulation for Decentralized Federated Learning
  3. 2025Towards Collaborative Fairness in Federated Learning Under Imbalanced Covariate Shift
  4. 2024Federated Unlearning Model Recovery in Data with Skewed Label Distributions
  5. 2024KDk: A Defense Mechanism Against Label Inference Attacks in Vertical Federated Learning
  6. 2026FedSIR: Spectral Client Identification and Relabeling for Federated Learning with Noisy Labels
  7. 2023FedNoRo: Towards Noise-Robust Federated Learning by Addressing Class Imbalance and Label Noise Heterogeneity
  8. 2025Tackling Data Heterogeneity in Federated Learning through Knowledge Distillation with Inequitable Aggregation