# Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating

arXiv: https://arxiv.org/abs/1910.08234 · HTML: https://mlsanity.com/paper/1910.08234/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2019.

## Built on by (most cited first, 22 of 22)

- [On the Importance and Applicability of Pre-Training for Federated Learning](https://mlsanity.com/paper/2206.11488/) (2022)
- [Byzantine Machine Learning Made Easy by Resilient Averaging of Momentums](https://mlsanity.com/paper/2205.12173/) (2022)
- [Gradient-Leakage Resilient Federated Learning](https://mlsanity.com/paper/2107.01154/) (2021)
- [Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning](https://mlsanity.com/paper/2102.12920/) (2021)
- [Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoT](https://mlsanity.com/paper/2202.01512/) (2022)
- [Joint Optimization in Edge-Cloud Continuum for Federated Unsupervised Person Re-identification](https://mlsanity.com/paper/2108.06493/) (2021)
- [On Bridging Generic and Personalized Federated Learning for Image Classification](https://mlsanity.com/paper/2107.00778/) (2021)
- [Continual Learning for Peer-to-Peer Federated Learning: A Study on Automated Brain Metastasis Identification](https://mlsanity.com/paper/2204.13591/) (2022)
- [Optimizing Performance of Federated Person Re-identification: Benchmarking and Analysis](https://mlsanity.com/paper/2205.12144/) (2022)
- [Heterogeneous Federated Learning via Grouped Sequential-to-Parallel Training](https://mlsanity.com/paper/2201.12976/) (2022)
- [MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classes](https://mlsanity.com/paper/2404.09232/) (2024)
- [Metrizing Fairness](https://mlsanity.com/paper/2205.15049/) (2022)
- [Aggregate or Not? Exploring Where to Privatize in DNN Based Federated Learning Under Different Non-IID Scenes](https://mlsanity.com/paper/2107.11954/) (2021)
- [Personalized Federated Learning with Exact Stochastic Gradient Descent](https://mlsanity.com/paper/2202.09848/) (2022)
- [Gradient Masked Federated Optimization](https://mlsanity.com/paper/2104.10322/) (2021)
- [FedDIP: Federated Learning with Extreme Dynamic Pruning and Incremental Regularization](https://mlsanity.com/paper/2309.06805/) (2023)
- [Gradient-Congruity Guided Federated Sparse Training](https://mlsanity.com/paper/2405.01189/) (2024)
- [Bias-Free FedGAN: A Federated Approach to Generate Bias-Free Datasets](https://mlsanity.com/paper/2103.09876/) (2021)
- [Data Poisoning and Leakage Analysis in Federated Learning](https://mlsanity.com/paper/2409.13004/) (2024)
- [Leveraging Learning Metrics for Improved Federated Learning](https://mlsanity.com/paper/2309.00257/) (2023)
- [Traversal Learning: A Lossless And Efficient Distributed Learning Framework](https://mlsanity.com/paper/2504.07471/) (2025)
- [FedEve: On Bridging the Client Drift and Period Drift for Cross-device Federated Learning](https://mlsanity.com/paper/2508.14539/) (2025)

## Abstract (arXiv)

Federated learning (FL) aims to train machine learning models in the decentralized system consisting of an enormous amount of smart edge devices. Federated averaging (FedAvg), the fundamental algorithm in FL settings, proposes on-device training and model aggregation to avoid the potential heavy communication costs and privacy concerns brought by transmitting raw data. However, through theoretical analysis we argue that 1) the multiple steps of local updating will result in gradient biases and 2) there is an inconsistency between the expected target distribution and the optimization objectives following the training paradigm in FedAvg. To tackle these problems, we first propose an unbiased gradient aggregation algorithm with the keep-trace gradient descent and the gradient evaluation strategy. Then we introduce an additional controllable meta updating procedure with a small set of data samples, indicating the expected target distribution, to provide a clear and consistent optimization objective. Both the two improvements are model- and task-agnostic and can be applied individually or together. Experimental results demonstrate that the proposed methods are faster in convergence and achieve higher accuracy with different network architectures in various FL settings.