# LiveServe: Interaction-Aware Serving for Real-Time Omni-Modal LLMs

arXiv: https://arxiv.org/abs/2606.22983 · HTML: https://mlsanity.com/paper/2606.22983/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2026.

## Built on by (most cited first, 1 of 1)

- [DuplexCadence: Exact State and Execution from a Speech Model's Declared Timelines](https://mlsanity.com/paper/2609.35115/) (2026)

## Abstract (arXiv)

Realtime omni-modal LMs support speech-centric conversations where users stream inputs, hear generated audio, and interrupt freely. Existing Omni-LM serving systems still rely on throughput-oriented LLM scheduling and LRU KV offloading. These policies ignore audio playback and multi-turn reuse: they may generate tokens far beyond what users hear, wasting work after barge-in, and evict KV state needed in the next turn. LiveServe is an interaction-aware serving system for realtime Omni-LM interaction. It exposes playback progress, speech activity, and barge-in events to the serving pipeline. The scheduler prioritizes first-audio and near-underrun sessions while limiting generation beyond the playback frontier. The KV manager uses next-use-aware eviction and preloads likely-needed KV during user speech to hide reload latency. On vLLM-Omni, LiveServe improves realtime serving across two Omni-LMs and mixed workloads. It lowers P90 audio TTFP by 1.55× on average and up to 2.21×, while improving completed-request throughput by 1.15× on average and up to 1.56×, and moves most KV reload work off the next-turn critical path.