# Serving systems and request scheduling

Area: [Efficient LLM inference and serving](https://mlsanity.com/inference/index.md). HTML: https://mlsanity.com/inference/serving-systems-and-request-scheduling/

## Key facts

- Serving systems and request scheduling: 402 papers on arXiv since 2021 (data through 14 Oct 2026).
- 178 of its 402 mapped papers (44%) cite “DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving” (2024, arXiv 2401.09670).
- 153 of its 402 mapped papers (38%) cite “Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve” (2024, arXiv 2403.02310).
- 19 new papers 24 Sep–14 Oct.

## Most cited here

- [DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving](https://mlsanity.com/paper/2401.09670/) (2024), cited by 178 papers here
- [Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve](https://mlsanity.com/paper/2403.02310/) (2024), cited by 153 papers here
- [Llumnix: Dynamic Scheduling for Large Language Model Serving](https://mlsanity.com/paper/2406.03243/) (2024), cited by 72 papers here
- [Fast Distributed Inference Serving for Large Language Models](https://mlsanity.com/paper/2305.05920/) (2023), cited by 63 papers here
- [Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving](https://mlsanity.com/paper/2407.00079/) (2024), cited by 61 papers here
- [SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills](https://mlsanity.com/paper/2308.16369/) (2023), cited by 51 papers here
- [SpotServe: Serving Generative Large Language Models on Preemptible Instances](https://mlsanity.com/paper/2311.15566/) (2023), cited by 45 papers here
- [LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism](https://mlsanity.com/paper/2404.09526/) (2024), cited by 41 papers here
- [BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems](https://mlsanity.com/paper/2401.17644/) (2024), cited by 39 papers here
- [Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction](https://mlsanity.com/paper/2404.08509/) (2024), cited by 32 papers here

## New (24 Sep–14 Oct)

- [RadixFlow: Filling Execution Bubbles in Prefix-Cached Hybrid LLM Serving](https://mlsanity.com/paper/2610.18143/) (2026-10-13)
- [Lightweight Scheduling Policies for Continuously Batched LLM Serving Across GPU Architectures](https://mlsanity.com/paper/2610.15899/) (2026-10-11)
- [Context-Aware Precision Scheduling for Small Language Models](https://mlsanity.com/paper/2610.15640/) (2026-10-11)
- [The Price of Precision: A Calibrated Cost Model for Mixed-Precision LLM Serving](https://mlsanity.com/paper/2610.15629/) (2026-10-11)
- [Dissecting the Carbon-SLO Tension in Geo-Distributed LLM Serving](https://mlsanity.com/paper/2610.14571/) (2026-10-10)
- [Taking Tokenization off the Host for Agentic LLM Serving](https://mlsanity.com/paper/2610.13325/) (2026-10-09)
- [TokenRouter: Efficient Serving System for Token-Level LLM Routing](https://mlsanity.com/paper/2610.12242/) (2026-10-08)
- [Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design](https://mlsanity.com/paper/2610.05062/) (2026-10-04)
- [FluidPD: In-Place Elasticity for SLO-Aware Prefill-Decode Disaggregated LLM Serving](https://mlsanity.com/paper/2610.06917/) (2026-10-02)
- [Characterizing High Bandwidth Flash for LLM Serving](https://mlsanity.com/paper/2609.39131/) (2026-09-30)
- [Cascadia: A Control-Plane-Free Alternative to Hyperconverged AI Infrastructure](https://mlsanity.com/paper/2609.38697/) (2026-09-30)
- [SPLASH: Switching Parallel Layouts of Attention with Seamless Handoff for LLM Serving](https://mlsanity.com/paper/2609.37626/) (2026-09-29)
- [vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains](https://mlsanity.com/paper/2609.37062/) (2026-09-29)
- [Dynamic Flow, Static Graph: KV Cache Reuse for Efficient LLM Serving on Mobile NPUs](https://mlsanity.com/paper/2609.34727/) (2026-09-28)
- [DPS: Dual-Mode Precision LLM Serving with Semi-Unified Memory](https://mlsanity.com/paper/2609.34380/) (2026-09-28)
- [Does Execution Require Target KV Fidelity? A Mixed-Fidelity KV Runtime for LLM Serving](https://mlsanity.com/paper/2609.33536/) (2026-09-27)
- [EAServe: Encode-Aware Disaggregated Serving for Multimodal Large Language Models](https://mlsanity.com/paper/2609.31551/) (2026-09-25)
- [DynBranch: Speculative Subgraph Reuse for Dynamic Agentic LLM Serving](https://mlsanity.com/paper/2609.31047/) (2026-09-25)
- [Cross-Model Autoscaling for Shared LLM Serving](https://mlsanity.com/paper/2609.29160/) (2026-09-24)