# SeaLLM: Service-Aware and Latency-Optimized Resource Sharing for Large Language Model Inference

arXiv: https://arxiv.org/abs/2504.15720 · HTML: https://mlsanity.com/paper/2504.15720/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2025.

## Built on by (most cited first, 4 of 4)

- [Coordinated Scheduling for MoE LLM Serving](https://mlsanity.com/paper/2606.15177/) (2026)
- [Multi-Layer Scheduling for MoE-Based LLM Reasoning](https://mlsanity.com/paper/2602.21626/) (2026)
- [Scepsy: Serving Agentic Workflows Using Aggregate LLM Pipelines](https://mlsanity.com/paper/2604.15186/) (2026)
- [ARL-Tangram: Unleash the Resource Efficiency in Agentic Reinforcement Learning](https://mlsanity.com/paper/2603.13019/) (2026)

## Abstract (arXiv)

Large language models (LLMs) with different architectures and sizes have been developed. Serving each LLM with dedicated GPUs leads to resource waste and service inefficiency due to the varying demand of LLM requests. A common practice is to share multiple LLMs. However, existing sharing systems either do not consider the autoregressive pattern of LLM services, or only focus on improving the throughput, which impairs the sharing performance, especially the serving latency. We present SeaLLM, which enables service-aware and latency-optimized LLM sharing. SeaLLM improves the overall sharing performance by (1) a latency-optimized scheduling algorithm utilizing the characteristics of LLM services, (2) a placement algorithm to determine the placement plan and an adaptive replacement algorithm to decide the replacement interval, and (3) a unified key-value cache to share GPU memory among LLM services efficiently. Our evaluation under real-world traces and LLM services demonstrates that SeaLLM improves the normalized latency by up to 13.60×, the tail latency by up to 18.69×, and the SLO attainment by up to 3.64× compared to existing solutions.