mlsanityData to 14 Oct 2026

Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider

arXiv 2003.03423 ↗First posted 6 Mar 2020cs.DCcited by 66

Not on the map. Why?

Abstract (arXiv)

Function as a Service (FaaS) has been gaining popularity as a way to deploy computations to serverless backends in the cloud. This paradigm shifts the complexity of allocating and provisioning resources to the cloud provider, which has to provide the illusion of always-available resources (i.e., fast function invocations without cold starts) at the lowest possible resource cost. Doing so requires the provider to deeply understand the characteristics of the FaaS workload. Unfortunately, there has been little to no public information on these characteristics. Thus, in this paper, we first characterize the entire production FaaS workload of Azure Functions. We show for example that most functions are invoked very infrequently, but there is an 8-order-of-magnitude range of invocation frequencies. Using observations from our characterization, we then propose a practical resource management policy that significantly reduces the number of function coldstarts,while spending fewerresources than state-of-the-practice policies.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 10; 2024: 14; 2025: 16; 2026: 18.

66 papers.

Sort:

By area:

  1. 2023SpotServe: Serving Generative Large Language Models on Preemptible Instancescited by 87
  2. 2024BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systemscited by 87
  3. 2024ServerlessLLM: Low-Latency Serverless Inference for Large Language Modelscited by 80
  4. 2021Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenterscited by 31
  5. 2025ServeGen: Workload Characterization and Generation of Large Language Model Serving in Productioncited by 29
  6. 2024SkyServe: Serving AI Models across Regions and Clouds with Spot Instancescited by 20
  7. 2026Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Servingcited by 20
  8. 2023DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMscited by 18
  9. 2025HydraServe: Minimizing Cold Start Latency for Serverless LLM Serving in Public Cloudscited by 14
  10. 2022Treehouse: A Case For Carbon-Aware Datacenter Softwarecited by 12
  11. 2023SuperServe: Fine-Grained Inference Serving for Unpredictable Workloadscited by 12
  12. 2025TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocitycited by 12
38 more
  1. 2026Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scalecited by 11
  2. 2023How Does It Function? Characterizing Long-term Trends in Production Serverless Workloadscited by 10
  3. 2024DAM: Towards A Foundation Model for Time Series Forecastingcited by 10
  4. 2025EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Servingcited by 10
  5. 2025FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusterscited by 9
  6. 2022Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniquescited by 7
  7. 2025Trusted AI Agents in the Cloudcited by 6
  8. 2023A Deep Recurrent-Reinforcement Learning Method for Intelligent AutoScaling of Serverless Functionscited by 5
  9. 2025Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Managementcited by 5
  10. 2025WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Servingcited by 5
  11. 2024CascadeServe: Unlocking Model Cascades for Inference Servingcited by 4
  12. 2024Is the GPU Half-Empty or Half-Full? Practical Scheduling Techniques for LLMscited by 4
  13. 2025Predictive-LoRA: A Proactive and Fragmentation-Aware Serverless Inference System for LLMscited by 3
  14. 2026FleetOpt: Analytical Fleet Provisioning for LLM Inference with Compress-and-Route as Implementation Mechanismcited by 3
  15. 2022Learning Cooperative Oversubscription for Cloud by Chance-Constrained Multi-Agent Reinforcement Learningcited by 2
  16. 2023OptScaler: A Collaborative Framework for Robust Autoscaling in the Cloudcited by 2
  17. 2023On a Foundation Model for Operating Systemscited by 2
  18. 2024Learned Best-Effort LLM Servingcited by 2
  19. 2024LIFL: A Lightweight, Event-driven Serverless Platform for Federated Learningcited by 2
  20. 2025PipeBoost: Resilient Pipelined Architecture for Fast Serverless LLM Scalingcited by 2
  21. 2025Tangram: Accelerating Serverless LLM Loading through GPU Memory Reuse and Affinitycited by 2
  22. 2026RLHFless: Serverless Computing for Efficient RLHFcited by 2
  23. 2022Learning-Assisted Algorithm Unrolling for Online Optimization with Budget Constraintscited by 1
  24. 2023Managing Cold-start in The Serverless Cloud with Temporal Convolutional Networkscited by 1
  25. 2023Intelligence-Endogenous Management Platform for Computing and Network Convergencecited by 1
  26. 2024COIN: Chance-Constrained Imitation Learning for Uncertainty-aware Adaptive Resource Oversubscription Policycited by 1
  27. 2024A Framework for SLO, Carbon, and Wastewater-Aware Sustainable FaaS Cloud Platform Managementcited by 1
  28. 2025ServerlessLoRA: Enabling Low-Latency Serverless Multi-LoRA Servingcited by 1
  29. 2025Vulcan: Instance-specialized, Verifiable Systems Heuristics Through LLM-driven Searchcited by 1
  30. 2026Autopoiesis: A Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamicscited by 1
  31. 2026SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Servingcited by 1
  32. 2021Accelerating Serverless Computing by Harvesting Idle Resourcescited by 0
  33. 2022Performance Modeling of Metric-Based Serverless Computing Platformscited by 0
  34. 2022The Metaverse Data Deluge: What Can We Do About It?cited by 0
  35. 2023On-demand Cold Start Frequency Reduction with Off-Policy Reinforcement Learning in Serverless Computingcited by 0
  36. 2024Analytically-Driven Resource Management for Cloud-Native Microservicescited by 0
  37. 2024Shabari: Delayed Decision-Making for Faster and Efficient Serverless Functionscited by 0
  38. 2024PerOS: Personalized Self-Adapting Operating Systems in the Cloudcited by 0

50 most cited of 66 All 66 (JSON)