mlsanityData to 14 Oct 2026

Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning

arXiv 2606.10250 ↗First posted 8 Jun 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Class imbalance is a common problem in deep learning that severely degrades performance. In federated learning (FL), it is a critical factor contributing to non-identically distributed data (non-IID). Building on several previous attempts, we define and analyze imbalance issues in FL at three levels: inter-case, inter-class, and inter-client. Inter-case imbalance addresses the imbalance in every single class; inter-class imbalance compares the number of data between different classes. Inter-client imbalance represents different skewness of local data between clients. Based on these concepts, we propose FedBB, which consists of two main components: (1) Positive Negative Balanced (PNB) loss function addresses the inter-case and inter-class imbalances in local training, enhancing generalization on highly skewed local client datasets. It optimizes both multi-label and multi-class classifications by assigning higher weights to minority cases or classes. (2) Client Balanced Reweighting (CBR) reweights clients based on inter-client imbalance during model aggregation, giving greater weight to models trained on less skewed datasets. Various experiments on X-ray and natural image datasets demonstrate that FedBB outperforms other algorithms in both performance and efficiency. Additionally, it requires limited statistical information, which is beneficial for privacy protection. Through ablation studies, we proved that PNB loss and CBR independently contribute to performance. As FedBB aims to build a global model that accurately classifies all classes, it can serve as a baseline for the generic and personalized FL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 66 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

40 of 66 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  7. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  8. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  9. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  10. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  11. 2021Model-Contrastive Federated Learningcited by 766
  12. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
28 more
  1. 2020The Future of Digital Health with Federated Learningcited by 633
  2. 2021Towards Personalized Federated Learningcited by 464
  3. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  4. 2019Fair Resource Allocation in Federated Learningcited by 394
  5. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  6. 2020Federated Visual Classification with Real-World Data Distributioncited by 128
  7. 2019One-Shot Federated Learningcited by 125
  8. 2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
  9. 2016Range Loss for Deep Face Recognition with Long-tailcited by 111
  10. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 109
  11. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
  12. 2022Layer-wised Model Aggregation for Personalized Federated Learningcited by 96
  13. 2017The Devil is in the Tails: Fine-grained Classification in the Wildcited by 89
  14. 2017Deep Active Learning over the Long Tailcited by 84
  15. 2021Distributional Robustness Loss for Long-tail Learningcited by 84
  16. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  17. 2016Factors in Finetuning Deep Model for object detectioncited by 53
  18. 2020Continual Local Training for Better Initialization of Federated Modelscited by 34
  19. 2020GradAug: A New Regularization Method for Deep Neural Networkscited by 18
  20. 2022FedIIC: Towards Robust Federated Learning for Class-Imbalanced Medical Image Classificationcited by 18
  21. 2020Bridging the gap between Natural and Medical Images through Deep Colorizationcited by 11
  22. 2021Imbalanced Adversarial Training with Reweightingcited by 11
  23. 2023HyperFed: Hyperbolic Prototypes Exploration with Consistent Aggregation for Non-IID Data in Federated Learningcited by 8
  24. 2021Addressing Fairness, Bias and Class Imbalance in Machine Learning: the FBI-losscited by 6
  25. 2023Robust Representation Learning with Reliable Pseudo-labels Generation via Self-Adaptive Optimal Transport for Short Text Clusteringcited by 6
  26. 2021Handling Inter-class and Intra-class Imbalance in Class-imbalanced Learningcited by 5
  27. 2022ISFL: Federated Learning for Non-i.i.d. Data with Local Importance Samplingcited by 4
  28. 2023Joint Local Relational Augmentation and Global Nash Equilibrium for Federated Learning with Non-IID Datacited by 3

Built on by

Not cited yet.