Sangam: Efficiently Serving Diffusion LLMs with the AR Stack
On the map: Diffusion models → Discrete and text diffusion
Abstract (arXiv)
Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation. Their bidirectional attention prevents exact autoregressive-style KV caching, since committing one position shifts the KV activations of all others. Approximate caching techniques such as Fast-dLLM and dKV-Cache refresh KV activations repeatedly and reuse them across intervening decodes, inducing a repeated prefill/decode structure. This makes AR serving mechanisms relevant to dLLMs, but not directly applicable. dLLM decodes are block-sized rather than token-sized, prefills recur, and bidirectional attention precludes the chunked prefill mechanism used for stall-free colocated serving. We present Sangam, a serving system for cached dLLM inference. Sangam introduces a deficit token-budget scheduler that admits in-flight decodes first, admits whole indivisible prefills only when the accumulated token budget allows, and carries unused budget forward. This achieves amortized stall-free scheduling. Disaggregated serving avoids prefill-decode interference but suffers from prefill/decode resource partitioning problem. Sangam adopts a hybrid serving strategy, overflowing prefills onto decode workers to relieve prefill under-provisioning, and uses the same deficit-budget scheduler to protect those workers' decodes from the overflow. We show that like AR serving, dLLM serving design space is governed by prefill-decode interference and prefill/decode partitioning. Colocated serving is most effective on decode-heavy workloads, cutting mean latency by 9-20% over hybrid execution on LLaDA-8B ShareGPT; while hybrid execution is most effective on prefill-heavy workloads, cutting mean latency by 8-20% over colocated execution on Dream-7B arXiv. Sangam is available at https://github.com/UT-InfraAI/sangam.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 34 of its 42 references that are in the map. First posted on arXiv 5 Jul 2026.
Builds on
34 of 42 references.
Closest ancestors · 9
Same direction, newest first
- 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
- 2025Soft-Masked Diffusion Language Models20 Oct
- 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
- 2025Dream-Coder 7B: An Open Diffusion Language Model for Code1 Sep
- 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
- 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
- 2024Scaling up Masked Diffusion Models on Text24 Oct
- 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
- 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
Other references · 25
- 2017Attention Is All You Needcited by 59,932
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
- 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
- 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
- 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
- 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 507
- 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
13 more
- 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
- 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 248
- 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
- 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
- 2024Llumnix: Dynamic Scheduling for Large Language Model Servingcited by 167
- 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
- 2024Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloadscited by 120
- 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 115
- 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 108
- 2025Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principlescited by 43
- 2025Beyond the Buzz: A Pragmatic Take on Inference Disaggregationcited by 9
- 2025Enabling Approximate Joint Sampling in Diffusion LMscited by 9
- 2025Taming the Memory Footprint Crisis: System Design for Production Diffusion LLM Servingcited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
- 2026Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM
- 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
- 2026Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
- 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
- 2026Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
- 2026CAI-DLLM: Convergence Aware Inference for Diffusion Language Models
- 2026NPU Design for Diffusion Language Model Inference