mlsanityData to 14 Oct 2026

Ascendra: Dynamic Request Prioritization for Efficient LLM Serving

arXiv 2504.20828 ↗First posted 29 Apr 2025cs.AIcited by 4

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

The rapid advancement of Large Language Models (LLMs) has driven the need for more efficient serving strategies. In this context, efficiency refers to the proportion of requests that meet their Service Level Objectives (SLOs), particularly for Time To First Token (TTFT) and Time Between Tokens (TBT). However, existing systems often prioritize one metric at the cost of the other. We present Ascendra, an LLM serving system designed to meet both TTFT and TBT SLOs simultaneously. The core insight behind Ascendra is that a request's urgency evolves as it approaches its deadline. To leverage this, Ascendra partitions GPU resources into two types of instances: low-priority and high-priority. Low-priority instances maximize throughput by processing requests out of arrival order, but at the risk of request starvation. To address this, Ascendra employs a performance model to predict requests at risk of missing their SLOs and proactively offloads them to high-priority instances. High-priority instances are optimized for low-latency execution and handle urgent requests nearing their deadlines. This partitioned architecture enables Ascendra to effectively balance high throughput and low latency. Extensive evaluation shows that Ascendra improves system throughput by up to 1.7x compared to vLLM and Sarathi-Serve while meeting both TTFT and TBT SLOs.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 14 of its 26 references that are in the map. First posted on arXiv 29 Apr 2025.

Builds on

14 of 26 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024NanoFlow: Towards Optimal Large Language Model Serving Throughput22 Aug
  2. 2024Llumnix: Dynamic Scheduling for Large Language Model Serving5 Jun
  3. 2024LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism15 Apr
  4. 2023Fairness in Serving Large Language Models31 Dec
  5. 2023SpotServe: Serving Generative Large Language Models on Preemptible Instances27 Nov

Other references · 9

  1. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  2. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  3. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  4. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 252
  5. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 203
  6. 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentioncited by 163
  7. 2024DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inferencecited by 106
  8. 2024ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Cachingcited by 48
  9. 2024ExeGPT: Constraint-Aware Resource Scheduling for LLM Inferencecited by 45

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Embodied Task Planning via Graph-Informed Action Generation with Large Language Modelscited by 2
  2. 2026Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Modelcited by 2
  3. 2026SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Schedulingcited by 2
  4. 2026Hierarchical Reranking for Scalable Financial RAG Systemcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
  2. 2025ProServe: Unified Multi-Priority Request Scheduling for LLM Serving
  3. 2025Scorpio: Serving Right Requests at the Right Time for Heterogeneous SLOs in LLM Inference
  4. 2025SLOs-Serve: Optimized Serving of Multi-SLO LLMs
  5. 2025TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Scheduling
  6. 2026Nitsum: Serving Tiered LLM Requests with Adaptive Tensor Parallelism
  7. 2026Beyond Binary Priorities: Multi-Tier SLA Scheduling for Large Language Model Serving
  8. 2024Queue management for slo-oriented large language model serving