# FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reduction

arXiv: https://arxiv.org/abs/2205.13462 · HTML: https://mlsanity.com/paper/2205.13462/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 52 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 30 of 37)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [Federated Learning with Matched Averaging](https://mlsanity.com/paper/2002.06440/) (2020)
- [Towards Personalized Federated Learning](https://mlsanity.com/paper/2103.00710/) (2021)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Agnostic Federated Learning](https://mlsanity.com/paper/1902.00146/) (2019)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [Escaping the Big Data Paradigm with Compact Transformers](https://mlsanity.com/paper/2104.05704/) (2021)
- [No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data](https://mlsanity.com/paper/2106.05001/) (2021)
- [Don't Use Large Mini-Batches, Use Local SGD](https://mlsanity.com/paper/1808.07217/) (2018)
- [Improve Unsupervised Domain Adaptation with Mixup Training](https://mlsanity.com/paper/2001.00677/) (2020)
- [Federated Continual Learning with Weighted Inter-client Transfer](https://mlsanity.com/paper/2003.03196/) (2020)
- [Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning](https://mlsanity.com/paper/2008.03606/) (2020)
- [Federated Adversarial Domain Adaptation](https://mlsanity.com/paper/1911.02054/) (2019)
- [FedMix: Approximation of Mixup under Mean Augmented Federated Learning](https://mlsanity.com/paper/2107.00233/) (2021)
- [Distributionally Robust Federated Averaging](https://mlsanity.com/paper/2102.12660/) (2021)
- [Preservation of the Global Knowledge by Not-True Distillation in Federated Learning](https://mlsanity.com/paper/2106.03097/) (2021)

## Built on by (most cited first, 30 of 35)

- [Relaxed Contrastive Learning for Federated Learning](https://mlsanity.com/paper/2401.04928/) (2024)
- [Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shifts](https://mlsanity.com/paper/2312.02567/) (2023)
- [FedImpro: Measuring and Improving Client Update in Federated Learning](https://mlsanity.com/paper/2402.07011/) (2024)
- [Do We Really Need to Design New Byzantine-robust Aggregation Rules?](https://mlsanity.com/paper/2501.17381/) (2025)
- [Filling the Missing: Exploring Generative AI for Enhanced Federated Learning over Heterogeneous Mobile Edge Devices](https://mlsanity.com/paper/2310.13981/) (2023)
- [FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning](https://mlsanity.com/paper/2501.05496/) (2025)
- [Lightweight Unsupervised Federated Learning with Pretrained Vision Language Model](https://mlsanity.com/paper/2404.11046/) (2024)
- [Navigating Distribution Shifts in Medical Image Analysis: A Survey](https://mlsanity.com/paper/2411.05824/) (2024)
- [Tackling the Non-IID Issue in Heterogeneous Federated Learning by Gradient Harmonization](https://mlsanity.com/paper/2309.06692/) (2023)
- [Faster Multi-GPU Training with PPLL: A Pipeline Parallelism Framework Leveraging Local Learning](https://mlsanity.com/paper/2411.12780/) (2024)
- [Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologies](https://mlsanity.com/paper/2310.05397/) (2023)
- [FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentation](https://mlsanity.com/paper/2312.02327/) (2023)
- [Uncertainty-Based Extensible Codebook for Discrete Federated Learning in Heterogeneous Data Silos](https://mlsanity.com/paper/2402.18888/) (2024)
- [Efficient Federated Learning against Byzantine Attacks and Data Heterogeneity via Aggregating Normalized Gradients](https://mlsanity.com/paper/2408.09539/) (2024)
- [FedBM: Stealing Knowledge from Pre-trained Language Models for Heterogeneous Federated Learning](https://mlsanity.com/paper/2502.16832/) (2025)
- [Operator-Theoretic Framework for Gradient-Free Federated Learning](https://mlsanity.com/paper/2512.01025/) (2025)
- [FedMABA: Towards Fair Federated Learning through Multi-Armed Bandits Allocation](https://mlsanity.com/paper/2410.20141/) (2024)
- [High-Energy Concentration for Federated Learning in Frequency Domain](https://mlsanity.com/paper/2509.12630/) (2025)
- [Distribution-Controlled Client Selection to Improve Federated Learning Strategies](https://mlsanity.com/paper/2509.20877/) (2025)
- [Beyond Aggregation: Guiding Clients in Heterogeneous Federated Learning](https://mlsanity.com/paper/2509.23049/) (2025)
- [H+: An Efficient Similarity-Aware Aggregation for Byzantine Resilient Federated Learning](https://mlsanity.com/paper/2509.24330/) (2025)
- [Personalized Collaborative Learning with Affinity-Based Variance Reduction](https://mlsanity.com/paper/2510.16232/) (2025)
- [Client2Vec: Improving Federated Learning by Distribution Shifts Aware Client Indexing](https://mlsanity.com/paper/2405.16233/) (2024)
- [FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentation](https://mlsanity.com/paper/2406.09547/) (2024)
- [An Architecture Built for Federated Learning: Addressing Data Heterogeneity through Adaptive Normalization-Free Feature Recalibration](https://mlsanity.com/paper/2410.02006/) (2024)
- [F-RBA: A Federated Learning-based Framework for Risk-based Authentication](https://mlsanity.com/paper/2412.12324/) (2024)
- [PQFed: A Privacy-Preserving Quality-Controlled Federated Learning Framework](https://mlsanity.com/paper/2509.21704/) (2025)
- [FedDPC : Handling Data Heterogeneity and Partial Client Participation in Federated Learning](https://mlsanity.com/paper/2512.20329/) (2025)
- [When the Server Steps In: Calibrated Updates for Fair Federated Learning](https://mlsanity.com/paper/2601.05352/) (2026)
- [Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection](https://mlsanity.com/paper/2602.01039/) (2026)

## Abstract (arXiv)

Federated Learning (FL) is a way for machines to learn from data that is kept locally, in order to protect the privacy of clients. This is typically done using local SGD, which helps to improve communication efficiency. However, such a scheme is currently constrained by slow and unstable convergence due to the variety of data on different clients' devices. In this work, we identify three under-explored phenomena of biased local learning that may explain these challenges caused by local updates in supervised FL. As a remedy, we propose FedBR, a novel unified algorithm that reduces the local learning bias on features and classifiers to tackle these challenges. FedBR has two components. The first component helps to reduce bias in local classifiers by balancing the output of the models. The second component helps to learn local features that are similar to global features, but different from those learned from other data sources. We conducted several experiments to test algopt and found that it consistently outperforms other SOTA FL methods. Both of its components also individually show performance gains. Our code is available at https://github.com/lins-lab/fedbr.