mlsanityData to 7 Oct 2026

Epoch: Compiling Diffusion Blocks for Sparse MoE Serving

arXiv 2609.09748 ↗First posted 9 Sep 2026cs.DCcited by 0

On the map: Mixture-of-experts → Vision mixture-of-experts

also in 1 more
Abstract (arXiv)

Diffusion language models generate text by refining a fixed-size block of token positions through many forward passes, a loop that does not match the per-forward execution unit used by most LLM serving systems. A dense MoE runtime binds all work to the refinement-iteration clock: it rebuilds similar routing structure on every forward, recomputes expert outputs for positions whose logits are already dead, and sends those positions through dense expert-parallel collectives. This paper presents sys, a serving system that treats the diffusion block as a compilation unit. sys compiles a small block plan for the block-clock structure of one diffusion block and refreshes every value that can affect a live decode decision on the iteration clock. sys realizes this plan along three dense axes of an MoE forward: atlas compiles a coverage-driven active expert support per layer while recomputing gate logits every iteration; lsp keeps full sequence shards as model state but routes only live, newly decoded, and refresh-required positions through fresh routed-expert computation; freshlane carries this fresh token--expert worklist through expert-parallel dispatch, kernels, and combine, then restores the dense logical shard at the layer boundary. We implement sys on 8 NVIDIA H100 GPUs and evaluate it on three open-weight block-diffusion MoE models (LLaDA-MoE, LLaDA2.0-mini, and LLaDA2.0-Flash, spanning 7B to 100B total parameters) across GSM8K, HumanEval, MGSM, and MT-Bench. sys improves end-to-end execution time by up to 2.7× over the strongest surviving baseline under the same 8-GPU placement and remains feasible at the largest batch sizes where multiple baselines run out of memory, while preserving task quality relative to the dense reference.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 34 of its 54 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

34 of 54 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  10. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  11. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  12. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
22 more
  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  2. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  3. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  4. 2019Root Mean Square Layer Normalizationcited by 1,425
  5. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  6. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  7. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  8. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  9. 2025Large Language Diffusion Modelscited by 872
  10. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  11. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  12. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  13. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  14. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  15. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 507
  16. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  17. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 359
  18. 2023PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUcited by 210
  19. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Expertscited by 204
  20. 2022Tutel: Adaptive Mixture-of-Experts at Scalecited by 199
  21. 2022Accelerating Distributed MoE Training and Inference with Linacited by 109
  22. 2025MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Productioncited by 28

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Model
  2. 2026TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Acceleration
  3. 2026Faster Block-Diffusion Serving with Distribution-Free Risk Guarantees
  4. 2026REFLEX: Rethinking MoE Inference as Refinement-Aware Compute Allocation in Diffusion Language Models
  5. 2026Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding
  6. 2025On Powerful Ways to Generate: Autoregression, Diffusion, and Beyond
  7. 2026Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Models
  8. 2026Sangam: Efficiently Serving Diffusion LLMs with the AR Stack