mlsanityData to 7 Oct 2026

DARE: Diffusion Language Model Activation Reuse for Efficient Inference

arXiv 2605.08134 ↗First posted 1 May 2026cs.LG cs.AIcited by 1

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to auto-regressive (AR) models, offering greater expressive capacity and potential for parallel generation and faster inference. However, open-source dLLMs remain immature, lagging behind AR models in both efficiency and quality. We identify an underexplored property of dLLMs: *token-wise redundancy* in bi-directional self-attention. Self-attention activations are highly correlated across tokens, and temporal changes in query representations can predict redundancy in corresponding key, value, and output activations. We introduce DARE, with two complementary mechanisms: DARE-KV, which reuses cached key-value (KV) activations, and DARE-O, which reuses output activations to reduce redundant computation while preserving quality. DARE achieves up to 1.20x per-layer latency reduction and reuses up to 87% of attention activations, with negligible degradation on reasoning and code-generation benchmarks. DARE-KV and DARE-O incur average performance drops of only 2.0% and 1.2%, respectively. Combined with techniques such as prefix caching and Fast-dLLM, DARE provides additive gains without retraining. These results establish token-wise reuse as an effective strategy for improving the efficiency of diffusion-based LLMs while preserving generation fidelity. Code: https://github.com/enyac-group/DARE

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 19 of its 22 references that are in the map. First posted on arXiv 1 May 2026.

Builds on

19 of 22 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Dream-Coder 7B: An Open Diffusion Language Model for Code1 Sep
  2. 2025Dream 7B: Diffusion Large Language Models21 Aug
  3. 2025DPad: Efficient Diffusion Language Models with Suffix Dropout19 Aug
  4. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  5. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  6. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  7. 2025Large Language Diffusion Models14 Feb

Other references · 12

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2024Mixtral of Expertscited by 1,785
  5. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  6. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  7. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  8. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
  9. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  10. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  11. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 59
  12. 2024MoDeGPT: Modular Decomposition for Large Language Model Compressioncited by 45

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Affix Cache for Diffusion Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026DARE: Diffusion Large Language Models Alignment and Reinforcement Executor
  2. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques
  3. 2026Retrofitting Linear Attention into Diffusion Language Models
  4. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference
  5. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
  6. 2026ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping
  7. 2025Activation Steering for Masked Diffusion Language Models
  8. 2025Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference