mlsanityData to 14 Oct 2026

ServerlessT2I: Efficient Text-to-Image Workflow Serving on a Serverless Platform

arXiv 2607.26566 ↗First posted 29 Jul 2026cs.DC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Text-to-image (T2I) workflows are increasingly deployed on serverless platforms because users often compose customized workflows and invoke them intermittently. Existing platforms typically deploy each workflow as an opaque GPU function, provisioning, placing, and scaling all constituent models in the workflow together. This monolithic design obscures workflow structure, inflates scaling overhead, forces users to manage low-level GPU coordination, and limits fine-grained fairness in multi-tenant clusters. In this paper, we present ServerlessT2I, a serverless-native system that decomposes a T2I workflow into loosely coupled model functions that can be independently managed and scheduled. By explicitly managing individual model execution, ServerlessT2I enables per-model scaling, declarative workflow composition, transparent GPU-resident communication, and fairness-aware scheduling. To make this decomposition efficient, ServerlessT2I harvests slack GPU memory left idle by compute-bound T2I inference to build a data plane that reduces model loading and data communication overheads. sys further introduces a fair scheduler for multi-tenant serving. Using production traces, ServerlessT2I sustains up to 2× higher request rates than existing T2I workflow serving systems with the same GPU budget; for a fixed request rate, it saves up to 3× GPU resources while satisfying service level objectives (SLOs).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 93 references that are in the map. First posted on arXiv 29 Jul 2026.

Builds on

29 of 93 references.

References · 29

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  3. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  4. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  5. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  6. 2021VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalizationcited by 287
  7. 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 277
  8. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  9. 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 202
  10. 2016Clipper: A Low-Latency Online Prediction Serving Systemcited by 191
  11. 2024OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-oncited by 151
  12. 2020Serving DNNs like Clockwork: Performance Predictability from the Bottom Upcited by 120
17 more
  1. 2024LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelismcited by 112
  2. 2024DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Modelscited by 98
  3. 2020Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Providercited by 66
  4. 2023Stateful Large Language Model Serving with Pensievecited by 61
  5. 2024xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelismcited by 56
  6. 2024Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flowcited by 55
  7. 2024MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Servingcited by 52
  8. 2025Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inferencecited by 22
  9. 2025RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Servingcited by 19
  10. 2023DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMscited by 18
  11. 2025DeepServe: Serverless Large Language Model Serving at Scalecited by 18
  12. 2017Occupy the Cloud: Distributed Computing for the 99%cited by 17
  13. 2025JITServe: SLO-aware LLM Serving with Imprecise Request Informationcited by 17
  14. 2022MGG: Accelerating Graph Neural Networks with Fine-grained intra-kernel Communication-Computation Pipelining on Multi-GPU Platformscited by 16
  15. 2023Torpor: GPU-Enabled Serverless Computing for Low-Latency, Resource-Efficient Inferencecited by 13
  16. 2025IC-Cache: Efficient Large Language Model Serving via In-context Cachingcited by 13
  17. 2024KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Servingcited by 11

Built on by

Not cited yet.