# Architecting Peer-to-Peer Serverless Distributed Machine Learning Training for Improved Fault Tolerance

arXiv: https://arxiv.org/abs/2302.13995 · HTML: https://mlsanity.com/paper/2302.13995/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 15 references that are in the map.
- First posted on arXiv 27 Feb 2023.

## Builds on (most cited first, 7 of 7)

- [A Survey on Distributed Machine Learning](https://mlsanity.com/paper/1912.09789/) (2019)
- [Towards Demystifying Serverless Machine Learning Training](https://mlsanity.com/paper/2105.07806/) (2021)
- [FedLess: Secure and Scalable Federated Learning Using Serverless Computing](https://mlsanity.com/paper/2111.03396/) (2021)
- [On Misbehaviour and Fault Tolerance in Machine Learning Systems](https://mlsanity.com/paper/2109.07989/) (2021)
- [SMLT: A Serverless Framework for Scalable and Adaptive Machine Learning Design and Training](https://mlsanity.com/paper/2205.01853/) (2022)
- [MLLess: Achieving Cost Efficiency in Serverless Machine Learning Training](https://mlsanity.com/paper/2206.05786/) (2022)
- [Costless: Optimizing Cost of Serverless Computing through Function Fusion and Placement](https://mlsanity.com/paper/1811.09721/) (2018)

## Abstract (arXiv)

Distributed Machine Learning refers to the practice of training a model on multiple computers or devices that can be called nodes. Additionally, serverless computing is a new paradigm for cloud computing that uses functions as a computational unit. Serverless computing can be effective for distributed learning systems by enabling automated resource scaling, less manual intervention, and cost reduction. By distributing the workload, distributed machine learning can speed up the training process and allow more complex models to be trained. Several topologies of distributed machine learning have been established (centralized, parameter server, peer-to-peer). However, the parameter server architecture may have limitations in terms of fault tolerance, including a single point of failure and complex recovery processes. Moreover, training machine learning in a peer-to-peer (P2P) architecture can offer benefits in terms of fault tolerance by eliminating the single point of failure. In a P2P architecture, each node or worker can act as both a server and a client, which allows for more decentralized decision making and eliminates the need for a central coordinator. In this position paper, we propose exploring the use of serverless computing in distributed machine learning training and comparing the performance of P2P architecture with the parameter server architecture, focusing on cost reduction and fault tolerance.