# FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems

arXiv: https://arxiv.org/abs/2104.07515 · HTML: https://mlsanity.com/paper/2104.07515/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 25 of 25)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Agnostic Federated Learning](https://mlsanity.com/paper/1902.00146/) (2019)
- [Fair Resource Allocation in Federated Learning](https://mlsanity.com/paper/1905.10497/) (2019)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning](https://mlsanity.com/paper/1807.06629/) (2018)
- [Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data](https://mlsanity.com/paper/1811.11479/) (2018)
- [Expanding the Reach of Federated Learning by Reducing Client Resource Requirements](https://mlsanity.com/paper/1812.07210/) (2018)
- [Communication Efficient Distributed Optimization using an Approximate Newton-type Method](https://mlsanity.com/paper/1312.7853/) (2013)
- [Communication Compression for Decentralized Training](https://mlsanity.com/paper/1803.06443/) (2018)
- [FedDANE: A Federated Newton-Type Method](https://mlsanity.com/paper/2001.01920/) (2020)
- [Astraea: Self-balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applications](https://mlsanity.com/paper/1907.01132/) (2019)
- [Active Federated Learning](https://mlsanity.com/paper/1909.12641/) (2019)
- [Federated Learning for Ultra-Reliable Low-Latency V2V Communications](https://mlsanity.com/paper/1805.09253/) (2018)
- [Robust and Communication-Efficient Collaborative Learning](https://mlsanity.com/paper/1907.10595/) (2019)
- [LoAdaBoost: loss-based AdaBoost federated machine learning with reduced computational complexity on IID and non-IID intensive care data](https://mlsanity.com/paper/1811.12629/) (2018)
- [Anytime MiniBatch: Exploiting Stragglers in Online Distributed Optimization](https://mlsanity.com/paper/2006.05752/) (2020)

## Built on by (most cited first, 16 of 16)

- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [Resource-Efficient Federated Learning](https://mlsanity.com/paper/2111.01108/) (2021)
- [A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications](https://mlsanity.com/paper/2403.01387/) (2024)
- [Personalized Federated Learning for Spatio-Temporal Forecasting: A Dual Semantic Alignment-Based Contrastive Approach](https://mlsanity.com/paper/2404.03702/) (2024)
- [A Survey of Federated Evaluation in Federated Learning](https://mlsanity.com/paper/2305.08070/) (2023)
- [ACE: A Model Poisoning Attack on Contribution Evaluation Methods in Federated Learning](https://mlsanity.com/paper/2405.20975/) (2024)
- [Flashback: Understanding and Mitigating Forgetting in Federated Learning](https://mlsanity.com/paper/2402.05558/) (2024)
- [Stabilizing and Improving Federated Learning with Non-IID Data and Client Dropout](https://mlsanity.com/paper/2303.06314/) (2023)
- [DPP-based Client Selection for Federated Learning with Non-IID Data](https://mlsanity.com/paper/2303.17358/) (2023)
- [Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectives](https://mlsanity.com/paper/2307.02140/) (2023)
- [PFL-GAN: When Client Heterogeneity Meets Generative Models in Personalized Federated Learning](https://mlsanity.com/paper/2308.12454/) (2023)
- [Personalized Recommendation Models in Federated Settings: A Survey](https://mlsanity.com/paper/2504.07101/) (2025)
- [FedLoRA-Optimizer: Federated LoRA Fine-Tuning with Global and Local Optimization in Heterogeneous Data Scenarios](https://mlsanity.com/paper/2510.11274/) (2025)
- [FedAgg: Adaptive Federated Learning with Aggregated Gradients](https://mlsanity.com/paper/2303.15799/) (2023)
- [Heterogeneity: An Open Challenge for Federated On-board Machine Learning](https://mlsanity.com/paper/2408.06903/) (2024)
- [EvoCSFL: Surrogate-Assisted Evolutionary Client Selection for Efficient and Robust Federated Learning](https://mlsanity.com/paper/2606.07702/) (2026)

## Abstract (arXiv)

Federated Learning (FL) is a novel distributed machine learning which allows thousands of edge devices to train model locally without uploading data concentrically to the server. But since real federated settings are resource-constrained, FL is encountered with systems heterogeneity which causes a lot of stragglers directly and then leads to significantly accuracy reduction indirectly. To solve the problems caused by systems heterogeneity, we introduce a novel self-adaptive federated framework FedSAE which adjusts the training task of devices automatically and selects participants actively to alleviate the performance degradation. In this work, we 1) propose FedSAE which leverages the complete information of devices' historical training tasks to predict the affordable training workloads for each device. In this way, FedSAE can estimate the reliability of each device and self-adaptively adjust the amount of training load per client in each round. 2) combine our framework with Active Learning to self-adaptively select participants. Then the framework accelerates the convergence of the global model. In our framework, the server evaluates devices' value of training based on their training loss. Then the server selects those clients with bigger value for the global model to reduce communication overhead. The experimental result indicates that in a highly heterogeneous system, FedSAE converges faster than FedAvg, the vanilla FL framework. Furthermore, FedSAE outperforms than FedAvg on several federated datasets - FedSAE improves test accuracy by 26.7% and reduces stragglers by 90.3% on average.