# Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction

arXiv: https://arxiv.org/abs/2209.15245 · HTML: https://mlsanity.com/paper/2209.15245/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Builds on (most cited first, 27 of 27)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://mlsanity.com/paper/1902.03932/) (2019)
- [Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies](https://mlsanity.com/paper/2010.01243/) (2020)
- [SCAFFOLD: Stochastic Controlled Averaging for Federated Learning](https://mlsanity.com/paper/1910.06378/) (2019)
- [Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning](https://mlsanity.com/paper/2105.05883/) (2021)
- [Astraea: Self-balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applications](https://mlsanity.com/paper/1907.01132/) (2019)
- [Towards Fair Federated Learning with Zero-Shot Data Augmentation](https://mlsanity.com/paper/2104.13417/) (2021)
- [Active Federated Learning](https://mlsanity.com/paper/1909.12641/) (2019)
- [An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guarantee](https://mlsanity.com/paper/2011.01783/) (2020)
- [Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity](https://mlsanity.com/paper/2012.14453/) (2020)
- [Federated learning with class imbalance reduction](https://mlsanity.com/paper/2011.11266/) (2020)
- [FLOP: Federated Learning on Medical Datasets using Partial Networks](https://mlsanity.com/paper/2102.05218/) (2021)
- [Communication-Efficient Federated Learning via Optimal Client Sampling](https://mlsanity.com/paper/2007.15197/) (2020)
- [Stochastic Particle-Optimization Sampling and the Non-Asymptotic Convergence Theory](https://mlsanity.com/paper/1809.01293/) (2018)
- [On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo](https://mlsanity.com/paper/1802.05431/) (2018)
- [FedMAX: Mitigating Activation Divergence for Accurate and Communication-Efficient Federated Learning](https://mlsanity.com/paper/2004.03657/) (2020)
- [Variance Reduction in Stochastic Particle-Optimization Sampling](https://mlsanity.com/paper/1811.08052/) (2018)
- [Self-Adversarially Learned Bayesian Sampling](https://mlsanity.com/paper/1811.08929/) (2018)

## Built on by (most cited first, 30 of 32)

- [Towards Building the Federated GPT: Federated Instruction Tuning](https://mlsanity.com/paper/2305.05644/) (2023)
- [A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advances](https://mlsanity.com/paper/2505.17799/) (2025)
- [SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Models](https://mlsanity.com/paper/2411.02433/) (2024)
- [FLrce: Resource-Efficient Federated Learning with Early-Stopping Strategy](https://mlsanity.com/paper/2310.09789/) (2023)
- [FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data](https://mlsanity.com/paper/2401.08977/) (2024)
- [FedMuon: Federated Learning with Bias-corrected LMO-based Optimization](https://mlsanity.com/paper/2509.26337/) (2025)
- [Meta-Reinforcement Learning via Exploratory Task Clustering](https://mlsanity.com/paper/2302.07958/) (2023)
- [Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspective](https://mlsanity.com/paper/2405.20431/) (2024)
- [SCALA: Split Federated Learning with Concatenated Activations and Logit Adjustments](https://mlsanity.com/paper/2405.04875/) (2024)
- [ARTIST: Improving the Generation of Text-rich Images with Disentangled Diffusion Models and Large Language Models](https://mlsanity.com/paper/2406.12044/) (2024)
- [MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning](https://mlsanity.com/paper/2409.06067/) (2024)
- [Federated Client-tailored Adapter for Medical Image Segmentation](https://mlsanity.com/paper/2504.18020/) (2025)
- [FLIPS: Federated Learning using Intelligent Participant Selection](https://mlsanity.com/paper/2308.03901/) (2023)
- [FedDD: Toward Communication-efficient Federated Learning with Differential Parameter Dropout](https://mlsanity.com/paper/2308.16835/) (2023)
- [Heterogeneity-Guided Client Sampling: Towards Fast and Efficient Non-IID Federated Learning](https://mlsanity.com/paper/2310.00198/) (2023)
- [Maverick-Aware Shapley Valuation for Client Selection in Federated Learning](https://mlsanity.com/paper/2405.12590/) (2024)
- [SFedCA: Credit Assignment-Based Active Client Selection Strategy for Spiking Federated Learning](https://mlsanity.com/paper/2406.12200/) (2024)
- [FedCGD: Collective Gradient Divergence Optimized Scheduling for Wireless Federated Learning](https://mlsanity.com/paper/2506.07581/) (2025)
- [Widening the Network Mitigates the Impact of Data Heterogeneity on FedAvg](https://mlsanity.com/paper/2508.12576/) (2025)
- [LEFL: Low Entropy Client Sampling in Federated Learning](https://mlsanity.com/paper/2312.17430/) (2023)
- [FedDiverse: Tackling Data Heterogeneity in Federated Learning with Diversity-Driven Client Selection](https://mlsanity.com/paper/2504.11216/) (2025)
- [Bridging Generalization Gap of Heterogeneous Federated Clients Using Generative Models](https://mlsanity.com/paper/2508.01669/) (2025)
- [FedTeddi: Temporal Drift and Divergence Aware Scheduling for Timely Federated Edge Learning](https://mlsanity.com/paper/2509.07342/) (2025)
- [Navigating High-Degree Heterogeneity: Federated Learning in Aerial and Space Networks](https://mlsanity.com/paper/2406.17951/) (2024)
- [FedProphet: Memory-Efficient Federated Adversarial Training via Robust and Consistent Cascade Learning](https://mlsanity.com/paper/2409.08372/) (2024)
- [BACSA: A Bias-Aware Client Selection Algorithm for Privacy-Preserving Federated Learning in Wireless Healthcare Networks](https://mlsanity.com/paper/2411.01050/) (2024)
- [Tackling Heterogeneity in Federated Learning via Variance-Reduced Boltzmann Sampling within Homogeneous Social Coalitions](https://mlsanity.com/paper/2506.02897/) (2025)
- [Communication-Efficient Federated Learning with Adaptive Number of Participants](https://mlsanity.com/paper/2508.13803/) (2025)
- [Federated Vision Transformer with Adaptive Focal Loss for Medical Image Classification](https://mlsanity.com/paper/2602.01633/) (2026)
- [Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition](https://mlsanity.com/paper/2602.19385/) (2026)

## Abstract (arXiv)

Due to limited communication capacities of edge devices, most existing federated learning (FL) methods randomly select only a subset of devices to participate in training for each communication round. Compared with engaging all the available clients, the random-selection mechanism can lead to significant performance degradation on non-IID (independent and identically distributed) data. In this paper, we show our key observation that the essential reason resulting in such performance degradation is the class-imbalance of the grouped data from randomly selected clients. Based on our key observation, we design an efficient heterogeneity-aware client sampling mechanism, i.e., Federated Class-balanced Sampling (Fed-CBS), which can effectively reduce class-imbalance of the group dataset from the intentionally selected clients. In particular, we propose a measure of class-imbalance and then employ homomorphic encryption to derive this measure in a privacy-preserving way. Based on this measure, we also design a computation-efficient client sampling strategy, such that the actively selected clients will generate a more class-balanced grouped dataset with theoretical guarantees. Extensive experimental results demonstrate Fed-CBS outperforms the status quo approaches. Furthermore, it achieves comparable or even better performance than the ideal setting where all the available clients participate in the FL training.