mlsanityData to 14 Oct 2026

Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale

arXiv 2609.11998 ↗First posted 9 Sep 2026cs.LGcited by 2

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

Diffusion language models decode tokens in parallel, but their bidirectional denoiser rules out the naive key--value (KV) cache behind fast autoregressive inference. Block diffusion restores caching by decoding block-by-block, and the block caches deployed on it so far are tied to attention: O(L)in memory and, if used as training-free retrofits, only an approximation of the model's computation. Both constraints can be overcome: sequence mixers that summarize finalized blocks into a reusable state support block caching, and the corresponding block-causal training objective makes the cache exact. We study this recipe at scale, pretraining three 3B block-diffusion denoisers (attention, mamba, and hybrid) on 300B tokens under one single-frontier objective and decoding all three through a single cached interface. Only the state-space cache is O(1) in sequence length: its memory and per-step latency stay constant at any context length, while an attention cache remains O(L). At 256k tokens (where attention has grown to 82GB and 29 ms/step), the Mamba cache delivers 4.3x lower latency, 11x less memory, and 2.6x higher single-stream throughput; and because that footprint is constant it scales with batch as well, reaching 14x the aggregate throughput, where attention cannot run beyond a single stream. The same linear-state bias lets the Mamba and hybrid backbones keep retrieving out to 8-16x their training length, whereas attention's retrieval collapses at 2x, at no measured quality cost.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 40 of its 41 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

40 of 41 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026Training Hybrid Block Diffusion Language Models with Partial Bidirectionality2 Jul
  2. 2026Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models9 Jun
  3. 2025DiffuMamba: High-Throughput Diffusion LMs with Mamba Backbone19 Nov
  4. 2025Diffusion Language Models are Super Data Learners5 Nov
  5. 2025Dream 7B: Diffusion Large Language Models21 Aug
  6. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  7. 2025LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs17 Jun
  8. 2025The Diffusion Duality12 Jun
  9. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  10. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  11. 2025Large Language Diffusion Models14 Feb
  12. 2024Simple and Effective Masked Diffusion Language Models11 Jun
2 more
  1. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  2. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  5. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  7. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  8. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  9. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  10. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  11. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 499
  12. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
14 more
  1. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
  2. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  3. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 401
  4. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 393
  5. 2024Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Modelscited by 242
  6. 2024An Empirical Study of Mamba-based Language Modelscited by 186
  7. 2024Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modelingcited by 174
  8. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  9. 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 160
  10. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139
  11. 2025A Systematic Analysis of Hybrid Linear Attentioncited by 39
  12. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
  13. 2025UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Modelscited by 13
  14. 2026Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusingcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Modelscited by 0
  2. 2026RL Post-Training Diffusion Language Models Along Their Own Generation Pathscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026One Block Ahead: Promoting Sampler State in Diffusion Language Models
  2. 2026Mask-Guided KV Cache Eviction in Block Diffusion Language Models
  3. 2026Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models
  4. 2025No Cache Left Idle: Accelerating diffusion model via Extreme-slimming Caching
  5. 2026FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion
  6. 2026Dynamic Chunking for Diffusion Language Models
  7. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Caching
  8. 2026Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation