mlsanityData to 14 Oct 2026

BestServe: Serving Strategies with Optimal Goodput in Collocation and Disaggregation Architectures

arXiv 2506.05871 ↗First posted 6 Jun 2025cs.LG cs.DC cs.PFcited by 3

Not on the map. Why?

Abstract (arXiv)

Serving large language models (LLMs) to millions of users requires efficient resource allocation and parallelism strategies. It is a labor intensive trial-and-error process to find such a strategy. We present BestServe, a novel framework for ranking serving strategies by estimating goodput under various operating scenarios. Supporting both collocated and disaggregated architectures, BestServe leverages an inference simulator built on an adapted roofline model and CPU-GPU dispatch dynamics. Our framework determines the optimal strategy in minutes on a single standard CPU, eliminating the need for costly benchmarking, while achieving predictions within a 20% error margin. It appeals to be practical for rapid deployment planning because of its lightweight design and strong extensibility.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 11 of its 28 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

11 of 28 references.

References · 11

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2016Layer Normalizationcited by 5,669
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  5. 2019Root Mean Square Layer Normalizationcited by 1,480
  6. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  7. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  8. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  9. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  10. 2024Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloadscited by 121
  11. 2018PipeDream: Fast and Efficient Pipeline Parallel DNN Trainingcited by 117

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025TimeBill: Time-Budgeted Inference for Large Language Modelscited by 4
  2. 2026Simple is Better: Multiplication May Be All You Need for LLM Request Schedulingcited by 2
  3. 2026Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inferencecited by 0