# Federated Learning with Fair Averaging

arXiv: https://arxiv.org/abs/2104.14937 · HTML: https://mlsanity.com/paper/2104.14937/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 23 of 23)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Agnostic Federated Learning](https://mlsanity.com/paper/1902.00146/) (2019)
- [Fair Resource Allocation in Federated Learning](https://mlsanity.com/paper/1905.10497/) (2019)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [The Non-IID Data Quagmire of Decentralized Machine Learning](https://mlsanity.com/paper/1910.00189/) (2019)
- [Survey of Personalization Techniques for Federated Learning](https://mlsanity.com/paper/2003.08673/) (2020)
- [Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms](https://mlsanity.com/paper/1808.07576/) (2018)
- [Collaborative Fairness in Federated Learning](https://mlsanity.com/paper/2008.12161/) (2020)
- [Mitigating Bias in Federated Learning](https://mlsanity.com/paper/2012.02447/) (2020)
- [Federated Learning Meets Multi-objective Optimization](https://mlsanity.com/paper/2006.11489/) (2020)
- [Bandit-based Communication-Efficient Client Selection Strategies for Federated Learning](https://mlsanity.com/paper/2012.08009/) (2020)
- [Fairness and Accuracy in Federated Learning](https://mlsanity.com/paper/2012.10069/) (2020)
- [Improved Schemes for Episodic Memory-based Lifelong Learning](https://mlsanity.com/paper/1909.11763/) (2019)
- [Towards Fair and Privacy-Preserving Federated Deep Models](https://mlsanity.com/paper/1906.01167/) (2019)

## Built on by (most cited first, 30 of 61)

- [FairFed: Enabling Group Fairness in Federated Learning](https://mlsanity.com/paper/2110.00857/) (2021)
- [Towards Fairness-Aware Federated Learning](https://mlsanity.com/paper/2111.01872/) (2021)
- [Recent Advances on Federated Learning: A Systematic Survey](https://mlsanity.com/paper/2301.01299/) (2023)
- [Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversity](https://mlsanity.com/paper/2405.16585/) (2024)
- [A Systematic Review on Long-Tailed Learning](https://mlsanity.com/paper/2408.00483/) (2024)
- [Minimax Demographic Group Fairness in Federated Learning](https://mlsanity.com/paper/2201.08304/) (2022)
- [No prejudice! Fair Federated Graph Neural Networks for Personalized Recommendation](https://mlsanity.com/paper/2312.10080/) (2023)
- [Improving Fairness in AI Models on Electronic Health Records: The Case for Federated Learning Methods](https://mlsanity.com/paper/2305.11386/) (2023)
- [Federated Unlearning with Gradient Descent and Conflict Mitigation](https://mlsanity.com/paper/2412.20200/) (2024)
- [Unified Group Fairness on Federated Learning](https://mlsanity.com/paper/2111.04986/) (2021)
- [AdaFed: Fair Federated Learning via Adaptive Common Descent Direction](https://mlsanity.com/paper/2401.04993/) (2024)
- [FLGo: A Fully Customizable Federated Learning Platform](https://mlsanity.com/paper/2306.12079/) (2023)
- [Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutions](https://mlsanity.com/paper/2406.10861/) (2024)
- [Federated Graph Learning for Cross-Domain Recommendation](https://mlsanity.com/paper/2410.08249/) (2024)
- [FedSampling: A Better Sampling Strategy for Federated Learning](https://mlsanity.com/paper/2306.14245/) (2023)
- [FedDRL: Deep Reinforcement Learning-based Adaptive Aggregation for Non-IID Data in Federated Learning](https://mlsanity.com/paper/2208.02442/) (2022)
- [Federated Learning with Domain Shift Eraser](https://mlsanity.com/paper/2503.13063/) (2025)
- [A Fair Federated Learning Framework With Reinforcement Learning](https://mlsanity.com/paper/2205.13415/) (2022)
- [FOCUS: Fairness via Agent-Awareness for Federated Learning on Heterogeneous Data](https://mlsanity.com/paper/2207.10265/) (2022)
- [CADIS: Handling Cluster-skewed Non-IID Data in Federated Learning with Clustered Aggregation and Knowledge DIStilled Regularization](https://mlsanity.com/paper/2302.10413/) (2023)
- [PraFFL: A Preference-Aware Scheme in Fair Federated Learning](https://mlsanity.com/paper/2404.08973/) (2024)
- [Distributed Quasi-Newton Method for Fair and Fast Federated Learning](https://mlsanity.com/paper/2501.10877/) (2025)
- [Entropy-driven Fair and Effective Federated Learning](https://mlsanity.com/paper/2301.12407/) (2023)
- [Federated Learning for 6G: Paradigms, Taxonomy, Recent Advances and Insights](https://mlsanity.com/paper/2312.04688/) (2023)
- [PUFFLE: Balancing Privacy, Utility, and Fairness in Federated Learning](https://mlsanity.com/paper/2407.15224/) (2024)
- [Federated Fairness Analytics: Quantifying Fairness in Federated Learning](https://mlsanity.com/paper/2408.08214/) (2024)
- [Adaptive Online Mirror Descent for Tchebycheff Scalarization in Multi-Objective Learning](https://mlsanity.com/paper/2410.21764/) (2024)
- [Joint Local Relational Augmentation and Global Nash Equilibrium for Federated Learning with Non-IID Data](https://mlsanity.com/paper/2308.11646/) (2023)
- [Linkage on Security, Privacy and Fairness in Federated Learning: New Balances and New Perspectives](https://mlsanity.com/paper/2406.10884/) (2024)
- [Contribution Evaluation of Heterogeneous Participants in Federated Learning via Prototypical Representations](https://mlsanity.com/paper/2407.02073/) (2024)

## Abstract (arXiv)

Fairness has emerged as a critical problem in federated learning (FL). In this work, we identify a cause of unfairness in FL -- conflicting gradients with large differences in the magnitudes. To address this issue, we propose the federated fair averaging (FedFV) algorithm to mitigate potential conflicts among clients before averaging their gradients. We first use the cosine similarity to detect gradient conflicts, and then iteratively eliminate such conflicts by modifying both the direction and the magnitude of the gradients. We further show the theoretical foundation of FedFV to mitigate the issue conflicting gradients and converge to Pareto stationary solutions. Extensive experiments on a suite of federated datasets confirm that FedFV compares favorably against state-of-the-art methods in terms of fairness, accuracy and efficiency. The source code is available at https://github.com/WwZzz/easyFL.