# Kairos: Low-latency Multi-Agent Serving with Shared LLMs and Excessive Loads in the Public Cloud

arXiv: https://arxiv.org/abs/2508.06948 · HTML: https://mlsanity.com/paper/2508.06948/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2025.

## Built on by (most cited first, 11 of 11)

- [CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control](https://mlsanity.com/paper/2601.22705/) (2026)
- [AgentCgroup: Understanding and Controlling OS Resources of AI Agents](https://mlsanity.com/paper/2602.09345/) (2026)
- [Learning Latency-Aware Orchestration for Multi-Agent Systems](https://mlsanity.com/paper/2601.10560/) (2026)
- [Characterizing Performance-Energy Trade-offs of Large Language Models in Multi-Request Workflows](https://mlsanity.com/paper/2604.09611/) (2026)
- [Benchmarking LLM Serving Systems for Agentic AI Workloads with XPerf](https://mlsanity.com/paper/2608.20370/) (2026)
- [TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving](https://mlsanity.com/paper/2608.25523/) (2026)
- [Alignment in Time: Peak-Aware Orchestration for Long-Horizon Agentic Systems](https://mlsanity.com/paper/2602.17910/) (2026)
- [Beyond Prediction: Tail-Aware Scheduling for LLM Inference](https://mlsanity.com/paper/2606.18431/) (2026)
- [Learning Latency-Aware Orchestration for Multi-Agent Systems](https://mlsanity.com/paper/2607.13359/) (2026)
- [Scalable LLM Agent Tool Access in the Cloud](https://mlsanity.com/paper/2607.15593/) (2026)
- [Workload-Aware Caching for Multi-Agent Systems](https://mlsanity.com/paper/2607.20495/) (2026)

## Abstract (arXiv)

Multi-agent applications utilize the advanced capabilities of large language models (LLMs) for intricate task completion through agent collaboration in a workflow. Under this situation, requests from different agents usually access the same shared LLM to perform different kinds of tasks, forcing the shared LLM to suffer excessive loads. However, existing works have low serving performance for these multi-agent applications, mainly due to the ignorance of inter-agent latency and resource differences for request scheduling. We therefore propose Kairos, a multi-agent orchestration system that optimizes end-to-end latency for multi-agent applications. Kairos consists of a workflow orchestrator, a workflow-aware priority scheduler, and a memory-aware dispatcher. The orchestrator collects agent-specific information for online workflow analysis. The scheduler decides the serving priority of the requests based on their latency characteristics to reduce the overall queuing. The dispatcher dispatches the requests to different LLM instances based on their memory demands to avoid GPU overloading. Experimental results show that Kairos reduces end-to-end latency by 17.8% to 28.4% compared to state-of-the-art works.