# Robust Model Aggregation for Heterogeneous Federated Learning: Analysis and Optimizations

arXiv: https://arxiv.org/abs/2405.06993 · HTML: https://mlsanity.com/paper/2405.06993/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 57 references that are in the map.
- First posted on arXiv 11 May 2024.

## Builds on (most cited first, 29 of 29)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Federated Learning with Differential Privacy: Algorithms and Performance Analysis](https://mlsanity.com/paper/1911.00222/) (2019)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning](https://mlsanity.com/paper/1807.06629/) (2018)
- [Asynchronous Federated Optimization](https://mlsanity.com/paper/1903.03934/) (2019)
- [Federated Learning with Buffered Asynchronous Aggregation](https://mlsanity.com/paper/2106.06639/) (2021)
- [Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective](https://mlsanity.com/paper/2004.04314/) (2020)
- [Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation](https://mlsanity.com/paper/1903.07424/) (2019)
- [Asynchronous Federated Learning on Heterogeneous Devices: A Survey](https://mlsanity.com/paper/2109.04269/) (2021)
- [SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overhead](https://mlsanity.com/paper/1910.01355/) (2019)
- [Towards Flexible Device Participation in Federated Learning](https://mlsanity.com/paper/2006.06954/) (2020)
- [FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers](https://mlsanity.com/paper/2010.05958/) (2020)
- [Blockchain Assisted Decentralized Federated Learning (BLADE-FL): Performance Analysis and Resource Allocation](https://mlsanity.com/paper/2101.06905/) (2021)
- [On Safeguarding Privacy and Security in the Framework of Federated Learning](https://mlsanity.com/paper/1909.06512/) (2019)
- [A Secure Federated Learning Framework for 5G Networks](https://mlsanity.com/paper/2005.05752/) (2020)
- [When Federated Learning Meets Blockchain: A New Distributed Learning Paradigm](https://mlsanity.com/paper/2009.09338/) (2020)
- [Low-Latency Federated Learning over Wireless Channels with Differential Privacy](https://mlsanity.com/paper/2106.13039/) (2021)
- [Trusted AI in Multi-agent Systems: An Overview of Privacy and Security for Distributed Learning](https://mlsanity.com/paper/2202.09027/) (2022)
- [Federated Learning in Intelligent Transportation Systems: Recent Applications and Open Problems](https://mlsanity.com/paper/2309.11039/) (2023)
- [Low-latency Federated Learning with DNN Partition in Distributed Industrial IoT Networks](https://mlsanity.com/paper/2210.14700/) (2022)
- [Blockchain Assisted Federated Learning over Wireless Channels: Dynamic Resource Allocation and Client Scheduling](https://mlsanity.com/paper/2105.14708/) (2021)
- [Writing Across the World's Languages: Deep Internationalization for Gboard, the Google Keyboard](https://mlsanity.com/paper/1912.01218/) (2019)
- [Secure Aggregation for Buffered Asynchronous Federated Learning](https://mlsanity.com/paper/2110.02177/) (2021)
- [Time-triggered Federated Learning over Wireless Networks](https://mlsanity.com/paper/2204.12426/) (2022)
- [Federated Learning Over Wireless Channels: Dynamic Resource Allocation and Task Scheduling](https://mlsanity.com/paper/2106.06934/) (2021)

## Abstract (arXiv)

Conventional synchronous federated learning (SFL) frameworks suffer from performance degradation in heterogeneous systems due to imbalanced local data size and diverse computing power on the client side. To address this problem, asynchronous FL (AFL) and semi-asynchronous FL have been proposed to recover the performance loss by allowing asynchronous aggregation. However, asynchronous aggregation incurs a new problem of inconsistency between local updates and global updates. Motivated by the issues of conventional SFL and AFL, we first propose a time-driven SFL (T-SFL) framework for heterogeneous systems. The core idea of T-SFL is that the server aggregates the models from different clients, each with varying numbers of iterations, at regular time intervals. To evaluate the learning performance of T-SFL, we provide an upper bound on the global loss function. Further, we optimize the aggregation weights to minimize the developed upper bound. Then, we develop a discriminative model selection (DMS) algorithm that removes local models from clients whose number of iterations falls below a predetermined threshold. In particular, this algorithm ensures that each client's aggregation weight accurately reflects its true contribution to the global model update, thereby improving the efficiency and robustness of the system. To validate the effectiveness of T-SFL with the DMS algorithm, we conduct extensive experiments using several popular datasets including MNIST, Cifar-10, Fashion-MNIST, and SVHN. The experimental results demonstrate that T-SFL with the DMS algorithm can reduce the latency of conventional SFL by 50%, while achieving an average 3% improvement in learning accuracy over state-of-the-art AFL algorithms.