# GENSERVE: Efficient Co-Serving of Heterogeneous Diffusion Model Workloads

arXiv: https://arxiv.org/abs/2604.04335 · HTML: https://mlsanity.com/paper/2604.04335/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2026.

## Built on by (most cited first, 4 of 4)

- [GF-DiT: Scheduling Parallelism for Diffusion Transformer Serving](https://mlsanity.com/paper/2606.13501/) (2026)
- [AoiZora: Topology-Aware Auto-Parallel Optimization for Inference of Diffusion Transformers](https://mlsanity.com/paper/2606.17566/) (2026)
- [Stateful Worlds, Stateless Elasticity: Exact-State Serving for Interactive World Models](https://mlsanity.com/paper/2607.10389/) (2026)
- [Efficient Vision-Language-Action Management and Serving for Robot Factories](https://mlsanity.com/paper/2609.12075/) (2026)

## Abstract (arXiv)

Diffusion models have emerged as the prevailing approach for text-to-image (T2I) and text-to-video (T2V) generation, yet production platforms must increasingly serve both modalities on shared GPU clusters while meeting stringent latency SLOs. Co-serving such heterogeneous workloads is challenging: T2I and T2V requests exhibit vastly different compute demands, parallelism characteristics, and latency requirements, leading to significant SLO violations in existing serving systems. We present GENSERVE, a co-serving system that leverages the inherent predictability of the diffusion process to optimize serving efficiency. A central insight is that diffusion inference proceeds in discrete, predictable steps and is naturally preemptible at step boundaries, opening a new design space for heterogeneity-aware resource management. GENSERVE introduces step-level resource adaptation through three coordinated mechanisms: intelligent video preemption, elastic sequence parallelism with dynamic batching, and an SLO-aware scheduler that jointly optimizes resource allocation across all concurrent requests. Experimental results show that GENSERVE improves the SLO attainment rate by up to 44% over the strongest baseline across diverse configurations.