mlsanityData to 14 Oct 2026

NPU Design for Diffusion Language Model Inference

arXiv 2601.20706 ↗First posted 28 Jan 2026cs.AR cs.AI cs.DCcited by 3

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

Diffusion-based LLMs (dLLMs) fundamentally depart from traditional autoregressive (AR) LLM inference: they leverage bidirectional attention, block-wise KV cache refreshing, cross-step reuse, and a non-GEMM-centric sampling phase. These characteristics make current dLLMs incompatible with most existing NPUs, as their inference patterns, in particular the reduction-heavy, top-k-driven sampling stage, demand new ISA and memory hierarchy support beyond that of AR accelerators. In addition, the blocked diffusion KV cache breaks from the append-only paradigm assumed by AR NPUs, and conventional AR-derived KV quantization schemes were designed for static activation distributions and do not account for the step-wise distribution shifts introduced by iterative block-wise refinement in dLLMs. In this paper, we introduce the first NPU accelerator specifically designed for dLLMs. It delivers: a dLLM-oriented ISA and compiler; a hardware-optimized execution model for both the transformer inference and diffusion sampling used in dLLMs; a novel Block-Adaptive Online Smoothing (BAOS) for quantizing KV cache in dLLMs; and a complete RTL implementation synthesized in 7nm. To evaluate and validate our design, we introduce a tri-path simulation framework that comprises analytical, cycle-accurate, and accuracy simulators, together with cross-validations against physical hardware. The full NPU stack, including ISA, simulation tools, and quantization software, will be open-sourced upon acceptance.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 14 of its 27 references that are in the map. First posted on arXiv 28 Jan 2026.

Builds on

14 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025dInfer: An Efficient Inference Framework for Diffusion Language Models9 Oct

Other references · 13

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  2. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  4. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  5. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  6. 2023Microscaling Data Formats for Deep Learningcited by 152
  7. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 92
  8. 2023Ramulator 2.0: A Modern, Modular, and Extensible DRAM Simulatorcited by 30
  9. 2024Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantizationcited by 28
  10. 2024MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantizationcited by 28
  11. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 23
  12. 2024ONNXim: A Fast, Cycle-level Multi-core NPU Simulatorcited by 5
1 more
  1. 2025P3-LLM: An Integrated NPU-PIM Accelerator for Edge LLM Inference Using Hybrid Numerical Formatscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026KernelCraft: Benchmarking for Agentic Close-to-Metal Kernel Generation on Emerging Hardwarecited by 5
  2. 2026Mask-Guided KV Cache Eviction in Block Diffusion Language Modelscited by 0
  3. 2026DynaTE: Accelerating Diffusion LLMs via Dynamic Token Executioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
  2. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
  3. 2026Sangam: Efficiently Serving Diffusion LLMs with the AR Stack
  4. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMs
  5. 2026CAI-DLLM: Convergence Aware Inference for Diffusion Language Models
  6. 2026ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping
  7. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
  8. 2025DiffuMamba: High-Throughput Diffusion LMs with Mamba Backbone