# Flexible Parallel Learning in Edge Scenarios: Communication, Computational and Energy Cost

arXiv: https://arxiv.org/abs/2201.07402 · HTML: https://mlsanity.com/paper/2201.07402/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 21 references that are in the map.
- First posted on arXiv 19 Jan 2022.

## Builds on (most cited first, 15 of 15)

- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [One weird trick for parallelizing convolutional neural networks](https://mlsanity.com/paper/1404.5997/) (2014)
- [A Joint Learning and Communications Framework for Federated Learning over Wireless Networks](https://mlsanity.com/paper/1909.07972/) (2019)
- [Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data](https://mlsanity.com/paper/1811.11479/) (2018)
- [Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing](https://mlsanity.com/paper/1905.10083/) (2019)
- [Federated Optimization:Distributed Optimization Beyond the Datacenter](https://mlsanity.com/paper/1511.03575/) (2015)
- [Reliable Federated Learning for Mobile Networks](https://mlsanity.com/paper/1910.06837/) (2019)
- [Network-Aware Optimization of Distributed Learning for Fog Computing](https://mlsanity.com/paper/2004.08488/) (2020)
- [Federated Learning at the Network Edge: When Not All Nodes are Created Equal](https://mlsanity.com/paper/2101.01995/) (2021)
- [Deep Learning Based Joint Resource Scheduling Algorithms for Hybrid MEC Networks](https://mlsanity.com/paper/1911.09598/) (2019)

## Built on by (most cited first, 1 of 1)

- [Federated Learning: A Cutting-Edge Survey of the Latest Advancements and Applications](https://mlsanity.com/paper/2310.05269/) (2023)

## Abstract (arXiv)

Traditionally, distributed machine learning takes the guise of (i) different nodes training the same model (as in federated learning), or (ii) one model being split among multiple nodes (as in distributed stochastic gradient descent). In this work, we highlight how fog- and IoT-based scenarios often require combining both approaches, and we present a framework for flexible parallel learning (FPL), achieving both data and model parallelism. Further, we investigate how different ways of distributing and parallelizing learning tasks across the participating nodes result in different computation, communication, and energy costs. Our experiments, carried out using state-of-the-art deep-network architectures and large-scale datasets, confirm that FPL allows for an excellent trade-off among computational (hence energy) cost, communication overhead, and learning performance.