# BARISTA: Efficient and Scalable Serverless Serving System for Deep Learning Prediction Services

arXiv: https://arxiv.org/abs/1904.01576 · HTML: https://mlsanity.com/paper/1904.01576/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2019.

## Built on by (most cited first, 6 of 6)

- [SkyServe: Serving AI Models across Regions and Clouds with Spot Instances](https://mlsanity.com/paper/2411.01438/) (2024)
- [Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision](https://mlsanity.com/paper/2205.11913/) (2022)
- [Distributed Double Machine Learning with a Serverless Architecture](https://mlsanity.com/paper/2101.04025/) (2021)
- [SMLT: A Serverless Framework for Scalable and Adaptive Machine Learning Design and Training](https://mlsanity.com/paper/2205.01853/) (2022)
- [Argus: Quality-Aware High-Throughput Text-to-Image Inference Serving System](https://mlsanity.com/paper/2511.06724/) (2025)
- [Unveiling the frontiers of deep learning: innovations shaping diverse domains](https://mlsanity.com/paper/2309.02712/) (2023)

## Abstract (arXiv)

Pre-trained deep learning models are increasingly being used to offer a variety of compute-intensive predictive analytics services such as fitness tracking, speech and image recognition. The stateless and highly parallelizable nature of deep learning models makes them well-suited for serverless computing paradigm. However, making effective resource management decisions for these services is a hard problem due to the dynamic workloads and diverse set of available resource configurations that have their deployment and management costs. To address these challenges, we present a distributed and scalable deep-learning prediction serving system called Barista and make the following contributions. First, we present a fast and effective methodology for forecasting workloads by identifying various trends. Second, we formulate an optimization problem to minimize the total cost incurred while ensuring bounded prediction latency with reasonable accuracy. Third, we propose an efficient heuristic to identify suitable compute resource configurations. Fourth, we propose an intelligent agent to allocate and manage the compute resources by horizontal and vertical scaling to maintain the required prediction latency. Finally, using representative real-world workloads for urban transportation service, we demonstrate and validate the capabilities of Barista.