# FedFa: A Fully Asynchronous Training Paradigm for Federated Learning

arXiv: https://arxiv.org/abs/2404.11015 · HTML: https://mlsanity.com/paper/2404.11015/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 17 Apr 2024.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Local SGD Converges Fast and Communicates Little](https://mlsanity.com/paper/1805.09767/) (2018)
- [Federated Learning with Buffered Asynchronous Aggregation](https://mlsanity.com/paper/2106.06639/) (2021)
- [Practical Secure Aggregation for Federated Learning on User-Held Data](https://mlsanity.com/paper/1611.04482/) (2016)
- [Asynchronous Federated Learning on Heterogeneous Devices: A Survey](https://mlsanity.com/paper/2109.04269/) (2021)
- [Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning](https://mlsanity.com/paper/2204.12703/) (2022)
- [Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning](https://mlsanity.com/paper/2002.04156/) (2020)
- [Perturbed Iterate Analysis for Asynchronous Stochastic Optimization](https://mlsanity.com/paper/1507.06970/) (2015)
- [Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics](https://mlsanity.com/paper/2110.01518/) (2021)
- [Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise](https://mlsanity.com/paper/2007.09208/) (2020)
- [A General Theory for Federated Optimization with Asynchronous and Heterogeneous Clients Updates](https://mlsanity.com/paper/2206.10189/) (2022)

## Built on by (most cited first, 12 of 12)

- [SyncFed: Time-Aware Federated Learning through Explicit Timestamping and Synchronization](https://mlsanity.com/paper/2506.09660/) (2025)
- [CLLoRA: An Approach to Measure the Effects of the Context Length for LLM Fine-Tuning](https://mlsanity.com/paper/2502.18910/) (2025)
- [Corrected with the Latest Version: Make Robust Asynchronous Federated Learning Possible](https://mlsanity.com/paper/2504.04081/) (2025)
- [AFBS:Buffer Gradient Selection in Semi-asynchronous Federated Learning](https://mlsanity.com/paper/2506.12754/) (2025)
- [Taming Latency and Bandwidth: A Theoretical Framework and Adaptive Algorithm for Communication-Constrained Training](https://mlsanity.com/paper/2507.17346/) (2025)
- [MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?](https://mlsanity.com/paper/2509.07727/) (2025)
- [SplitFT: An Adaptive Federated Split Learning System For LLMs Fine-Tuning](https://mlsanity.com/paper/2604.26388/) (2026)
- [Scalable Asynchronous Federated Modeling for Spatial Data](https://mlsanity.com/paper/2510.01771/) (2025)
- [FTTE: Enabling Federated and Resource-Constrained Deep Edge Intelligence](https://mlsanity.com/paper/2510.03165/) (2025)
- [FedPSA: Modeling Behavioral Staleness in Asynchronous Federated Learning](https://mlsanity.com/paper/2602.15337/) (2026)
- [FedQueue: Queue-Aware Federated Learning for Cross-Facility HPC Training](https://mlsanity.com/paper/2605.02125/) (2026)
- [FedRAW: Preserving Rare-Label Influence in Asynchronous Federated Learning](https://mlsanity.com/paper/2609.07192/) (2026)

## Abstract (arXiv)

Federated learning has been identified as an efficient decentralized training paradigm for scaling the machine learning model training on a large number of devices while guaranteeing the data privacy of the trainers. FedAvg has become a foundational parameter update strategy for federated learning, which has been promising to eliminate the effect of the heterogeneous data across clients and guarantee convergence. However, the synchronization parameter update barriers for each communication round during the training significant time on waiting, slowing down the training procedure. Therefore, recent state-of-the-art solutions propose using semi-asynchronous approaches to mitigate the waiting time cost with guaranteed convergence. Nevertheless, emerging semi-asynchronous approaches are unable to eliminate the waiting time completely. We propose a full asynchronous training paradigm, called FedFa, which can guarantee model convergence and eliminate the waiting time completely for federated learning by using a few buffered results on the server for parameter updating. Further, we provide theoretical proof of the convergence rate for our proposed FedFa. Extensive experimental results indicate our approach effectively improves the training performance of federated learning by up to 6x and 4x speedup compared to the state-of-the-art synchronous and semi-asynchronous strategies while retaining high accuracy in both IID and Non-IID scenarios.