# FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clients

arXiv: https://arxiv.org/abs/2201.01601 · HTML: https://mlsanity.com/paper/2201.01601/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 84 references that are in the map.
- First posted on arXiv 5 Jan 2022.

## Builds on (most cited first, 30 of 44)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [Federated Learning with Matched Averaging](https://mlsanity.com/paper/2002.06440/) (2020)
- [Federated Learning with Differential Privacy: Algorithms and Performance Analysis](https://mlsanity.com/paper/1911.00222/) (2019)
- [Flower: A Friendly Federated Learning Research Framework](https://mlsanity.com/paper/2007.14390/) (2020)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Fair Resource Allocation in Federated Learning](https://mlsanity.com/paper/1905.10497/) (2019)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients](https://mlsanity.com/paper/2010.01264/) (2020)
- [Not All Samples Are Created Equal: Deep Learning with Importance Sampling](https://mlsanity.com/paper/1803.00942/) (2018)
- [CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition](https://mlsanity.com/paper/2004.00288/) (2020)
- [On The Power of Curriculum Learning in Training Deep Networks](https://mlsanity.com/paper/1904.03626/) (2019)
- [Improving Federated Learning Personalization via Model Agnostic Meta Learning](https://mlsanity.com/paper/1909.12488/) (2019)
- [Automated Curriculum Learning for Neural Networks](https://mlsanity.com/paper/1704.03003/) (2017)
- [Applied Federated Learning: Improving Google Keyboard Query Suggestions](https://mlsanity.com/paper/1812.02903/) (2018)
- [Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies](https://mlsanity.com/paper/2010.01243/) (2020)
- [FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout](https://mlsanity.com/paper/2102.13451/) (2021)

## Built on by (most cited first, 30 of 46)

- [Client Selection in Federated Learning: Principles, Challenges, and Opportunities](https://mlsanity.com/paper/2211.01549/) (2022)
- [Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2404.06448/) (2024)
- [Recent Advances on Federated Learning: A Systematic Survey](https://mlsanity.com/paper/2301.01299/) (2023)
- [HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models](https://mlsanity.com/paper/2505.02795/) (2025)
- [FwdLLM: Efficient FedLLM using Forward Gradient](https://mlsanity.com/paper/2308.13894/) (2023)
- [FedAdapter: Efficient Federated Learning for Modern NLP](https://mlsanity.com/paper/2205.10162/) (2022)
- [FS-Real: Towards Real-World Cross-Device Federated Learning](https://mlsanity.com/paper/2303.13363/) (2023)
- [FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction](https://mlsanity.com/paper/2407.19389/) (2024)
- [Federated Few-Shot Learning for Mobile NLP](https://mlsanity.com/paper/2212.05974/) (2022)
- [DYNAMITE: Dynamic Interplay of Mini-Batch Size and Aggregation Frequency for Federated Learning with Static and Streaming Dataset](https://mlsanity.com/paper/2310.14906/) (2023)
- [MergeSFL: Split Federated Learning with Feature Merging and Batch Size Regulation](https://mlsanity.com/paper/2311.13348/) (2023)
- [FedTherapist: Mental Health Monitoring with User-Generated Linguistic Expressions on Smartphones via Federated Learning](https://mlsanity.com/paper/2310.16538/) (2023)
- [Not All Federated Learning Algorithms Are Created Equal: A Performance Evaluation Study](https://mlsanity.com/paper/2403.17287/) (2024)
- [Effective Heterogeneous Federated Learning via Efficient Hypernetwork-based Weight Generation](https://mlsanity.com/paper/2407.03086/) (2024)
- [A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications](https://mlsanity.com/paper/2403.01387/) (2024)
- [To Store or Not? Online Data Selection for Federated Learning with Limited Storage](https://mlsanity.com/paper/2209.00195/) (2022)
- [Flame: Simplifying Topology Extension in Federated Learning](https://mlsanity.com/paper/2305.05118/) (2023)
- [FedConv: A Learning-on-Model Paradigm for Heterogeneous Federated Clients](https://mlsanity.com/paper/2502.20639/) (2025)
- [Delta: A Cloud-assisted Data Enrichment Framework for On-Device Continual Learning](https://mlsanity.com/paper/2410.18378/) (2024)
- [Mitigating Group Bias in Federated Learning for Heterogeneous Devices](https://mlsanity.com/paper/2309.07085/) (2023)
- [Improving Local Training in Federated Learning via Temperature Scaling](https://mlsanity.com/paper/2401.09986/) (2024)
- [(FL)2: Overcoming Few Labels in Federated Semi-Supervised Learning](https://mlsanity.com/paper/2410.23227/) (2024)
- [Breaking the Memory Wall for Heterogeneous Federated Learning via Model Splitting](https://mlsanity.com/paper/2410.11577/) (2024)
- [Federated Learning in IoT: a Survey from a Resource-Constrained Perspective](https://mlsanity.com/paper/2308.13157/) (2023)
- [LIFL: A Lightweight, Event-driven Serverless Platform for Federated Learning](https://mlsanity.com/paper/2405.10968/) (2024)
- [Learning to Collaborate: An Orchestrated-Decentralized Framework for Peer-to-Peer LLM Federation](https://mlsanity.com/paper/2601.17133/) (2026)
- [DynamicFL: Balancing Communication Dynamics and Client Manipulation for Federated Learning](https://mlsanity.com/paper/2308.06267/) (2023)
- [EcoLearn: Optimizing the Carbon Footprint of Federated Learning](https://mlsanity.com/paper/2310.17972/) (2023)
- [FedHybrid: Breaking the Memory Wall of Federated Learning via Hybrid Tensor Management](https://mlsanity.com/paper/2510.11400/) (2025)
- [Knowledge Rumination for Client Utility Evaluation in Heterogeneous Federated Learning](https://mlsanity.com/paper/2312.10425/) (2023)

## Abstract (arXiv)

Federated Learning (FL) trains a machine learning model on distributed clients without exposing individual data. Unlike centralized training that is usually based on carefully-organized data, FL deals with on-device data that are often unfiltered and imbalanced. As a result, conventional FL training protocol that treats all data equally leads to a waste of local computational resources and slows down the global learning process. To this end, we propose FedBalancer, a systematic FL framework that actively selects clients' training samples. Our sample selection strategy prioritizes more "informative" data while respecting privacy and computational capabilities of clients. To better utilize the sample selection to speed up global training, we further introduce an adaptive deadline control scheme that predicts the optimal deadline for each round with varying client training data. Compared with existing FL algorithms with deadline configuration methods, our evaluation on five datasets from three different domains shows that FedBalancer improves the time-to-accuracy performance by 1.20~4.48x while improving the model accuracy by 1.1~5.0%. We also show that FedBalancer is readily applicable to other FL approaches by demonstrating that FedBalancer improves the convergence speed and accuracy when operating jointly with three different FL algorithms.