# Cornserve: A Distributed Serving System for Any-to-Any Multimodal Models

arXiv: https://arxiv.org/abs/2603.12118 · HTML: https://mlsanity.com/paper/2603.12118/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Builds on (most cited first, 17 of 17)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving](https://mlsanity.com/paper/2401.09670/) (2024)
- [Splitwise: Efficient generative LLM inference using phase splitting](https://mlsanity.com/paper/2311.18677/) (2023)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation](https://mlsanity.com/paper/2410.13848/) (2024)
- [LTX-2: Efficient Joint Audio-Visual Foundation Model](https://mlsanity.com/paper/2601.03233/) (2026)
- [Llumnix: Dynamic Scheduling for Large Language Model Serving](https://mlsanity.com/paper/2406.03243/) (2024)
- [xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism](https://mlsanity.com/paper/2411.01738/) (2024)
- [ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production](https://mlsanity.com/paper/2505.09999/) (2025)
- [Efficiently Serving Large Multimodal Models Using EPD Disaggregation](https://mlsanity.com/paper/2501.05460/) (2024)
- [Cornfigurator: Automated Planning for Any-to-Any Multimodal Model Serving](https://mlsanity.com/paper/2512.14098/) (2025)

## Built on by (most cited first, 1 of 1)

- [FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications](https://mlsanity.com/paper/2607.18171/) (2026)

## Abstract (arXiv)

Any-to-Any models are an emerging class of multimodal models that accept combinations of multimodal data (e.g., text, image, video, audio) as input and generate them as output. Serving these models are challenging; different requests with different input and output modalities traverse different paths through the model computation graph, and each component of the model have different scaling characteristics. We present Cornserve, a distributed serving system for generic Any-to-Any models. Cornserve provides a flexible task abstraction for expressing Any-to-Any model computation graphs, enabling component disaggregation and independent scaling. The distributed runtime dispatches compute to the data plane via an efficient record-and-replay execution model that keeps track of data dependencies, and forwards tensor data between components directly from the producer to the consumer. Built on Kubernetes with approximately 23K new lines of Python, Cornserve supports diverse Any-to-Any models and delivers up to 3.81× higher throughput and 5.79× lower tail latency. Cornserve is open-source, and the demo video is available on YouTube.