mlsanityData to 14 Oct 2026

ServerlessLoRA: Enabling Low-Latency Serverless Multi-LoRA Serving

arXiv 2505.14468 ↗First posted 20 May 2025cs.LG cs.DCcited by 1

Not on the map. Why?

Abstract (arXiv)

Multi-LoRA (Low-Rank Adaptation) serving allows many specialized LLM variants to share the same base model by attaching lightweight adapters. This makes it attractive for serving large catalogs of domain-, tenant-, and task-specific models. However, existing deployments waste resources on rarely used adapters, while directly running LoRA inference on serverless platforms introduces high memory overhead, repeated cold starts, and poor scheduling decisions. This paper presents ServerlessLoRA, a serverless multi-LoRA serving system that separates serving state into shared backbone state, reusable variant warm state, and request-private state. ServerlessLoRA loads each backbone once, shares it read-only across isolated LoRA functions, batches selected backbone operations, selectively warms variant state, and routes requests based on state locality and GPU contention. Evaluated on industrial traces, ServerlessLoRA reduces time-to-first-token by up to 92.3% against serverless baselines and achieves 1.66-3.01 times and 2.20-3.26 times higher latency-cost efficiency than vLLM-LoRA and dLoRA, respectively.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 33 of its 67 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

33 of 67 references.

References · 33

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  8. 2023Mistral 7Bcited by 3,164
  9. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  10. 2024Mixtral of Expertscited by 1,823
  11. 2024Phi-4 Technical Reportcited by 711
  12. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
21 more
  1. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  2. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  3. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
  4. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 365
  5. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 245
  6. 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 177
  7. 2023S-LoRA: Serving Thousands of Concurrent LoRA Adapterscited by 152
  8. 2024DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiencycited by 112
  9. 2024BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systemscited by 87
  10. 2023Punica: Multi-Tenant LoRA Servingcited by 83
  11. 2024ServerlessLLM: Low-Latency Serverless Inference for Large Language Modelscited by 80
  12. 2020Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Providercited by 66
  13. 2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
  14. 2024BLITZSCALE: Fast and Live Large Model Autoscaling with O(1) Host Cachingcited by 25
  15. 2025Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooningcited by 24
  16. 2025HydraServe: Minimizing Cold Start Latency for Serverless LLM Serving in Public Cloudscited by 14
  17. 2023Torpor: GPU-Enabled Serverless Computing for Low-Latency, Resource-Efficient Inferencecited by 13
  18. 2023FaST-GShare: Enabling Efficient Spatio-Temporal GPU Sharing in Serverless Computing for Deep Learning Inferencecited by 6
  19. 2025Dilu: Enabling GPU Resourcing-on-Demand for Serverless DL Serving via Introspective Elasticitycited by 3
  20. 2020InfiniCache: Exploiting Ephemeral Serverless Functions to Build a Cost-Effective Memory Cachecited by 2
  21. 2020Fifer: Tackling Underutilization in the Serverless Eracited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MoEless: Efficient MoE LLM Serving via Serverless Computingcited by 1