# The Cost of Training Machine Learning Models over Distributed Data Sources

arXiv: https://arxiv.org/abs/2209.07124 · HTML: https://mlsanity.com/paper/2209.07124/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 73 references that are in the map.
- First posted on arXiv 15 Sep 2022.

## Builds on (most cited first, 30 of 30)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection](https://mlsanity.com/paper/1907.09693/) (2019)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [Asynchronous Federated Optimization](https://mlsanity.com/paper/1903.03934/) (2019)
- [A Joint Learning and Communications Framework for Federated Learning over Wireless Networks](https://mlsanity.com/paper/1909.07972/) (2019)
- [Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing](https://mlsanity.com/paper/1905.10083/) (2019)
- [A Survey on Distributed Machine Learning](https://mlsanity.com/paper/1912.09789/) (2019)
- [Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models](https://mlsanity.com/paper/2007.03051/) (2020)
- [Mitigating Sybils in Federated Learning Poisoning](https://mlsanity.com/paper/1808.04866/) (2018)
- [Distributed Learning in Wireless Networks: Recent Progress and Future Challenges](https://mlsanity.com/paper/2104.02151/) (2021)
- [Blockchained On-Device Federated Learning](https://mlsanity.com/paper/1808.03949/) (2018)
- [Reliable Federated Learning for Mobile Networks](https://mlsanity.com/paper/1910.06837/) (2019)
- [Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence](https://mlsanity.com/paper/1909.00560/) (2019)
- [Federated Learning Meets Blockchain in Edge Computing: Opportunities and Challenges](https://mlsanity.com/paper/2104.01776/) (2021)
- [Wireless Communications for Collaborative Federated Learning](https://mlsanity.com/paper/2006.02499/) (2020)
- [An Energy and Carbon Footprint Analysis of Distributed and Federated Learning](https://mlsanity.com/paper/2206.10380/) (2022)
- [Continual Learning for Peer-to-Peer Federated Learning: A Study on Automated Brain Metastasis Identification](https://mlsanity.com/paper/2204.13591/) (2022)
- [FDA3 : Federated Defense Against Adversarial Attacks for Cloud-Based IIoT Applications](https://mlsanity.com/paper/2006.15632/) (2020)
- [IEEE 802.11ax: High-Efficiency WLANs](https://mlsanity.com/paper/1501.01496/) (2015)
- [BlockSim: An Extensible Simulation Tool for Blockchain Systems](https://mlsanity.com/paper/2004.13438/) (2020)
- [End-to-End Latency Analysis and Optimal Block Size of Proof-of-Work Blockchain Applications](https://mlsanity.com/paper/2202.01497/) (2022)
- [Green-PoW: An Energy-Efficient Blockchain Proof-of-Work Consensus Algorithm](https://mlsanity.com/paper/2007.04086/) (2020)
- [Discrete-Time Analysis of Wireless Blockchain Networks](https://mlsanity.com/paper/2104.05586/) (2021)

## Built on by (most cited first, 18 of 18)

- [The Future of Large Language Model Pre-training is Federated](https://mlsanity.com/paper/2405.10853/) (2024)
- [Federated Learning for 5G Base Station Traffic Forecasting](https://mlsanity.com/paper/2211.15220/) (2022)
- [RAZOR: Ratio-Aware Layer Editing for Targeted Unlearning in Vision Transformers and Diffusion Models](https://mlsanity.com/paper/2603.14819/) (2026)
- [Towards Energy-Aware Federated Traffic Prediction for Cellular Networks](https://mlsanity.com/paper/2309.10645/) (2023)
- [Federated Learning in Mobile Networks: A Comprehensive Case Study on Traffic Forecasting](https://mlsanity.com/paper/2412.04081/) (2024)
- [Distributed Learning for Wi-Fi AP Load Prediction](https://mlsanity.com/paper/2405.05140/) (2024)
- [Feature Aggregation with Latent Generative Replay for Federated Continual Learning of Socially Appropriate Robot Behaviours](https://mlsanity.com/paper/2405.15773/) (2024)
- [Energy-Aware Decentralized Learning with Intermittent Model Training](https://mlsanity.com/paper/2407.01283/) (2024)
- [Energy-Efficient Federated Learning for AIoT using Clustering Methods](https://mlsanity.com/paper/2505.09704/) (2025)
- [The Implications of Decentralization in Blockchained Federated Learning: Evaluating the Impact of Model Staleness and Inconsistencies](https://mlsanity.com/paper/2310.07471/) (2023)
- [A Declarative Query Language for Scientific Machine Learning](https://mlsanity.com/paper/2405.16159/) (2024)
- [AQMLator -- An Auto Quantum Machine Learning E-Platform](https://mlsanity.com/paper/2409.18338/) (2024)
- [Learn More by Using Less: Distributed Learning with Energy-Constrained Devices](https://mlsanity.com/paper/2412.02289/) (2024)
- [Evaluation of Bio-Inspired Models under Different Learning Settings For Energy Efficiency in Network Traffic Prediction](https://mlsanity.com/paper/2412.17565/) (2024)
- [Energy Minimization for Participatory Federated Learning in IoT Analyzed via Game Theory](https://mlsanity.com/paper/2503.21722/) (2025)
- [Stratify: Rethinking Federated Learning for Non-IID Data through Balanced Sampling](https://mlsanity.com/paper/2504.13462/) (2025)
- [Contrastive Self-Supervised Learning at the Edge: An Energy Perspective](https://mlsanity.com/paper/2510.08374/) (2025)
- [CARGO: Carbon-Aware Gossip Orchestration in Smart Shipping](https://mlsanity.com/paper/2603.27857/) (2026)

## Abstract (arXiv)

Federated learning is one of the most appealing alternatives to the standard centralized learning paradigm, allowing a heterogeneous set of devices to train a machine learning model without sharing their raw data. However, it requires a central server to coordinate the learning process, thus introducing potential scalability and security issues. In the literature, server-less federated learning approaches like gossip federated learning and blockchain-enabled federated learning have been proposed to mitigate these issues. In this work, we propose a complete overview of these three techniques proposing a comparison according to an integral set of performance indicators, including model accuracy, time complexity, communication overhead, convergence time, and energy consumption. An extensive simulation campaign permits to draw a quantitative analysis considering both feedforward and convolutional neural network models. Results show that gossip federated learning and standard federated solution are able to reach a similar level of accuracy, and their energy consumption is influenced by the machine learning model adopted, the software library, and the hardware used. Differently, blockchain-enabled federated learning represents a viable solution for implementing decentralized learning with a higher level of security, at the cost of an extra energy usage and data sharing. Finally, we identify open issues on the two decentralized federated learning implementations and provide insights on potential extensions and possible research directions in this new research field.