mlsanityData to 7 Oct 2026

Enabling KV Caching of Shared Prefix for Diffusion Language Models

arXiv 2606.07571 ↗First posted 26 May 2026cs.LG cs.AIcited by 1

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Key-value (KV) caching for shared prefixes is essential for high-throughput large language model (LLM) serving, but it faces critical challenges in emerging diffusion language models (DLMs). In DLMs, bidirectional attention means that updating any token dynamically alters the entire context and its corresponding KVs. Thus, existing caching techniques developed for LLMs, which assume that KVs remain invariant once computed, corrupt the shared prefix KVs. Our experiments show that applying these techniques to DLMs causes model accuracy to collapse to near zero. To unlock high-throughput DLM serving, we propose bidirectional prefix caching, BiCache, the first KV caching technique for shared prefixes in DLMs. BiCache is designed based on key observations from our comprehensive analysis: shared prefix KVs remain stable and reusable in shallow layers, while the depth of shallow layers depends on the fraction of shared prefix tokens in each request. Thus, BiCache dynamically identifies a safe layer depth for reusing shared prefix KVs and eliminates redundant computation. Evaluations demonstrate that BiCache significantly improves serving throughput by 36.3%-98.3% compared to existing techniques without accuracy collapse (only 0-1.8% difference).

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 27 of its 40 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

27 of 40 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference28 Dec
  2. 2025TiDAR: Think in Diffusion, Talk in Autoregression12 Nov
  3. 2025Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs31 Aug
  4. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  5. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion27 May

Other references · 22

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  4. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  5. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  6. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  7. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  8. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  9. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  10. 2023Birth of a Transformer: A Memory Viewpointcited by 154
  11. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  12. 2024Efficient LLM Scheduling by Learning to Rankcited by 83
10 more
  1. 2024Preble: Efficient Distributed Prompt Scheduling for LLM Servingcited by 68
  2. 2024Measuring and Controlling Instruction (In)Stability in Language Model Dialogscited by 51
  3. 2019Spherical Text Embeddingcited by 49
  4. 2024TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodputcited by 36
  5. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
  6. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
  7. 2024RelayAttention for Efficient Large Language Model Serving with Long System Promptscited by 13
  8. 2025Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Modelscited by 13
  9. 2026ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skippingcited by 6
  10. 2025LongWeave: A Long-Form Generation Benchmark Bridging Real-World Relevance and Verifiabilitycited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Affix Cache for Diffusion Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reuse
  2. 2026SPA-Cache: Singular Proxies for Adaptive Caching in Diffusion Language Models
  3. 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inference
  4. 2024A Systematic Study of Cross-Layer KV Sharing for Efficient LLM Inference
  5. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption
  6. 2024MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
  7. 2026EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models
  8. 2026A Universal Context-Reuse Layer for Cross-Model KV Sharing