# Enabling KV Caching of Shared Prefix for Diffusion Language Models

arXiv: https://arxiv.org/abs/2606.07571 · HTML: https://mlsanity.com/paper/2606.07571/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Discrete and text diffusion, one of 14 directions in Diffusion models.
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 26 May 2026.

## Placed in

- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 27 of 27)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding](https://mlsanity.com/paper/2505.22618/) (2025)
- [The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"](https://mlsanity.com/paper/2309.12288/) (2023)
- [LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models](https://mlsanity.com/paper/2505.19223/) (2025)
- [Prompt Cache: Modular Attention Reuse for Low-Latency Inference](https://mlsanity.com/paper/2311.04934/) (2023)
- [A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems](https://mlsanity.com/paper/2402.18013/) (2024)
- [dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching](https://mlsanity.com/paper/2506.06295/) (2025)
- [Birth of a Transformer: A Memory Viewpoint](https://mlsanity.com/paper/2306.00802/) (2023)
- [Fast-dLLM v2: Efficient Block-Diffusion LLM](https://mlsanity.com/paper/2509.26328/) (2025)
- [Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing](https://mlsanity.com/paper/2508.09192/) (2025)
- [Efficient LLM Scheduling by Learning to Rank](https://mlsanity.com/paper/2408.15792/) (2024)
- [FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion](https://mlsanity.com/paper/2505.21467/) (2025)
- [Preble: Efficient Distributed Prompt Scheduling for LLM Serving](https://mlsanity.com/paper/2407.00023/) (2024)
- [Measuring and Controlling Instruction (In)Stability in Language Model Dialogs](https://mlsanity.com/paper/2402.10962/) (2024)
- [Spherical Text Embedding](https://mlsanity.com/paper/1911.01196/) (2019)
- [WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference](https://mlsanity.com/paper/2512.22737/) (2025)
- [TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput](https://mlsanity.com/paper/2406.14066/) (2024)
- [TiDAR: Think in Diffusion, Talk in Autoregression](https://mlsanity.com/paper/2511.08923/) (2025)
- [d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation](https://mlsanity.com/paper/2601.07568/) (2026)
- [Attention Is All You Need for KV Cache in Diffusion LLMs](https://mlsanity.com/paper/2510.14973/) (2025)
- [RelayAttention for Efficient Large Language Model Serving with Long System Prompts](https://mlsanity.com/paper/2402.14808/) (2024)
- [Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models](https://mlsanity.com/paper/2510.04146/) (2025)
- [Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs](https://mlsanity.com/paper/2509.00707/) (2025)
- [ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping](https://mlsanity.com/paper/2603.10088/) (2026)
- [LongWeave: A Long-Form Generation Benchmark Bridging Real-World Relevance and Verifiability](https://mlsanity.com/paper/2510.24345/) (2025)

## Built on by (most cited first, 1 of 1)

- [Affix Cache for Diffusion Large Language Models](https://mlsanity.com/paper/2608.26140/) (2026)

## Abstract (arXiv)

Key-value (KV) caching for shared prefixes is essential for high-throughput large language model (LLM) serving, but it faces critical challenges in emerging diffusion language models (DLMs). In DLMs, bidirectional attention means that updating any token dynamically alters the entire context and its corresponding KVs. Thus, existing caching techniques developed for LLMs, which assume that KVs remain invariant once computed, corrupt the shared prefix KVs. Our experiments show that applying these techniques to DLMs causes model accuracy to collapse to near zero. To unlock high-throughput DLM serving, we propose bidirectional prefix caching, BiCache, the first KV caching technique for shared prefixes in DLMs. BiCache is designed based on key observations from our comprehensive analysis: shared prefix KVs remain stable and reusable in shallow layers, while the depth of shallow layers depends on the fraction of shared prefix tokens in each request. Thus, BiCache dynamically identifies a safe layer depth for reusing shared prefix KVs and eliminates redundant computation. Evaluations demonstrate that BiCache significantly improves serving throughput by 36.3%-98.3% compared to existing techniques without accuracy collapse (only 0-1.8% difference).