Scalability and Performance Evaluation of Federated Learning Frameworks: A Comparative Analysis
Not on the map. Why?
Abstract (arXiv)
This paper presents a systematic examination and experimental comparison of the prominent Federated Learning (FL) frameworks FedML, Flower, Substra, and OpenFL. The frameworks are evaluated experimentally by implementing Federated Learning over a varying number of clients, emphasizing a thorough analysis of scalability and key performance metrics. The study assesses the impact of increasing client counts on total training time, loss and accuracy values, and CPU and RAM usage. Results indicate distinct performance characteristics among the frameworks, with Flower displaying an unusually high loss, FedML achieving a notably low accuracy range of 66% to 79%, and Substra demonstrating good resource efficiency, albeit with an exponential growth in total training time. Notably, OpenFL emerges as the most scalable platform, demonstrating consistent accuracy, loss, and training time across different client counts. OpenFL's stable CPU and RAM underscore its reliability in real-world scenarios. This comprehensive analysis provides valuable insights into the relative performance of FL frameworks, offering good understanding of their capabilities and providing guidance for their effective deployment across diverse user bases.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 44 references that are in the map. First posted on arXiv 14 Sep 2026.
Builds on
17 of 44 references.
References · 17
- 2020Flower: A Friendly Federated Learning Research Frameworkcited by 455
- 2019Federated Machine Learning: Concept and Applicationscited by 412
- 2015Recent Advances in Convolutional Neural Networkscited by 360
- 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 300
- 2019A Joint Learning and Communications Framework for Federated Learning over Wireless Networkscited by 263
- 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
- 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
- 2022Vertical Federated Learning: Concepts, Advances and Challengescited by 108
- 2020Learning to Detect Malicious Clients for Robust Federated Learningcited by 88
- 2021OpenFL: An open-source framework for Federated Learningcited by 58
- 2020Wireless Communications for Collaborative Federated Learningcited by 47
- 2022Cross-Silo Federated Learning: Challenges and Opportunitiescited by 38
5 more
- 2020Federated Transfer Learning: concept and applicationscited by 32
- 2021The Role of Cross-Silo Federated Learning in Facilitating Data Sharing in the Agri-Food Sectorcited by 16
- 2019Substra: a framework for privacy-preserving, traceable and collaborative Machine Learningcited by 14
- 2022Federated learning: Applications, challenges and future directionscited by 11
- 2022A Practical Cross-Device Federated Learning Framework over 5G Networkscited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.