mlsanityData to 14 Oct 2026

Efficient On-Device Diffusion LLM Inference with Mobile NPU

arXiv 2606.13740 ↗First posted 11 Jun 2026cs.LGcited by 1

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Diffusion large language models (dLLMs) accelerate generation by denoising multiple tokens in parallel, making them attractive for latency-sensitive mobile inference. However, repeated denoising introduces substantial computation on smartphones. Mobile neural processing units (NPUs) offer high-throughput dense matrix computation, but efficiently exploiting them remains challenging: token commitment shrinks per-block effective workloads, token revision complicates KV cache reuse, and limited NPU-visible address space incurs costly remapping and data transfer overheads. In this paper, we propose llada.cpp, the first NPU-aware inference framework for accelerating dLLMs on smartphones. llada.cpp aligns block-wise dLLM inference with the execution characteristics of mobile NPUs through three techniques. (1) Multi-Block Speculative Decoding fills the shrinking workload in late-stage current-block decoding with speculative future-block tokens. (2) Dual-Path Progressive Revision keeps committed tokens revisable until stable and refreshes unstable tokens through a CPU-side path without stalling dense NPU execution. (3) Swap-Optimized Memory Runtime compacts NPU-visible address layouts and overlaps data staging with NPU computation to reduce remapping and transfer overheads. We implement llada.cpp as an end-to-end framework and evaluate it across diverse hardware platforms and dLLM workloads. llada.cpp reduces LLaDA-8B generation latency by 17x-42x over the CPU baseline with prefix KV cache reuse, while preserving generation quality.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 25 of its 42 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

25 of 42 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation12 Jan
  2. 2025dParallel: Learnable Parallel Decoding for dLLMs30 Sep
  3. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May

Other references · 22

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  3. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  5. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  7. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  8. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
  9. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  10. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 235
  11. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  12. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 189
10 more
  1. 2023AutoDroid: LLM-powered Task Automation in Androidcited by 184
  2. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  3. 2023MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devicescited by 122
  4. 2024Fast On-device LLM Inference with NPUscited by 62
  5. 2023Explore, Select, Derive, and Recall: Augmenting LLM with Human-like Memory for Mobile Task Automationcited by 58
  6. 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editingcited by 54
  7. 2025Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inferencecited by 22
  8. 2025Scaling LLM Test-Time Compute with Mobile NPU on Smartphonescited by 11
  9. 2026ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skippingcited by 6
  10. 2026FlashBlock: Attention Caching for Efficient Long-Context Block Diffusioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026EStream: Fast and Memory-Efficient MoE Prefill through Expert Virtualization on Mobile NPUscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026When NPUs Are Not Always Faster: A Stage-Level Analysis of Mobile LLM Inference
  2. 2026Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference
  3. 2026Lever: Speculative LLM Inference on Smartphones
  4. 2025MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices
  5. 2026Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
  6. 2026EdgeFlow: Fast Cold Starts for LLMs on Mobile Devices ↗
  7. 2026Quant.npu: Enabling Efficient Mobile NPU Inference for on-device LLMs via Fully Static Quantization
  8. 2026CAI-DLLM: Convergence Aware Inference for Diffusion Language Models