# SMLT: A Serverless Framework for Scalable and Adaptive Machine Learning Design and Training

arXiv: https://arxiv.org/abs/2205.01853 · HTML: https://mlsanity.com/paper/2205.01853/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 54 references that are in the map.
- First posted on arXiv 4 May 2022.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Horovod: fast and easy distributed deep learning in TensorFlow](https://mlsanity.com/paper/1802.05799/) (2018)
- [MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems](https://mlsanity.com/paper/1512.01274/) (2015)
- [cpSGD: Communication-efficient and differentially-private distributed SGD](https://mlsanity.com/paper/1805.10559/) (2018)
- [Chiron: Privacy-preserving Machine Learning as a Service](https://mlsanity.com/paper/1803.05961/) (2018)
- [Beating Atari with Natural Language Guided Reinforcement Learning](https://mlsanity.com/paper/1704.05539/) (2017)
- [MLModelCI: An Automatic Cloud Platform for Efficient MLaaS](https://mlsanity.com/paper/2006.05096/) (2020)
- [Towards Demystifying Serverless Machine Learning Training](https://mlsanity.com/paper/2105.07806/) (2021)
- [Packaging and Sharing Machine Learning Models via the Acumos AI Open Platform](https://mlsanity.com/paper/1810.07159/) (2018)
- [Golden Grain: Building a Secure and Decentralized Model Marketplace for MLaaS](https://mlsanity.com/paper/2011.06458/) (2020)
- [Bridging Data Center AI Systems with Edge Computing for Actionable Information Retrieval](https://mlsanity.com/paper/2105.13967/) (2021)
- [BARISTA: Efficient and Scalable Serverless Serving System for Deep Learning Prediction Services](https://mlsanity.com/paper/1904.01576/) (2019)
- [AI-based Resource Allocation: Reinforcement Learning for Adaptive Auto-scaling in Serverless Environments](https://mlsanity.com/paper/2005.14410/) (2020)
- [Semi-Dynamic Load Balancing: Efficient Distributed Learning in Non-Dedicated Environments](https://mlsanity.com/paper/1806.02508/) (2018)

## Built on by (most cited first, 4 of 4)

- [Exploring the Impact of Serverless Computing on Peer To Peer Training Machine Learning](https://mlsanity.com/paper/2309.14139/) (2023)
- [SPIRT: A Fault-Tolerant and Reliable Peer-to-Peer Serverless ML Training Architecture](https://mlsanity.com/paper/2309.14148/) (2023)
- [FedLesScan: Mitigating Stragglers in Serverless Federated Learning](https://mlsanity.com/paper/2211.05739/) (2022)
- [Architecting Peer-to-Peer Serverless Distributed Machine Learning Training for Improved Fault Tolerance](https://mlsanity.com/paper/2302.13995/) (2023)

## Abstract (arXiv)

In today's production machine learning (ML) systems, models are continuously trained, improved, and deployed. ML design and training are becoming a continuous workflow of various tasks that have dynamic resource demands. Serverless computing is an emerging cloud paradigm that provides transparent resource management and scaling for users and has the potential to revolutionize the routine of ML design and training. However, hosting modern ML workflows on existing serverless platforms has non-trivial challenges due to their intrinsic design limitations such as stateless nature, limited communication support across function instances, and limited function execution duration. These limitations result in a lack of an overarching view and adaptation mechanism for training dynamics and an amplification of existing problems in ML workflows. To address the above challenges, we propose SMLT, an automated, scalable, and adaptive serverless framework to enable efficient and user-centric ML design and training. SMLT employs an automated and adaptive scheduling mechanism to dynamically optimize the deployment and resource scaling for ML tasks during training. SMLT further enables user-centric ML workflow execution by supporting user-specified training deadlines and budget limits. In addition, by providing an end-to-end design, SMLT solves the intrinsic problems in serverless platforms such as the communication overhead, limited function execution duration, need for repeated initialization, and also provides explicit fault tolerance for ML training. SMLT is open-sourced and compatible with all major ML frameworks. Our experimental evaluation with large, sophisticated modern ML models demonstrate that SMLT outperforms the state-of-the-art VM based systems and existing serverless ML training frameworks in both training speed (up to 8X) and monetary cost (up to 3X)