Neither Parallel Nor Sequential: How DiffusionGemma Actually Commits Tokens
On the map: Diffusion models → Discrete and text diffusion
Abstract (arXiv)
Open diffusion language models are marketed as parallel, non-autoregressive decoders, yet the order in which a shipped checkpoint actually commits its tokens is almost never measured. We instrument DiffusionGemma 26B, a masked discrete-diffusion mixture-of-experts model built on Gemma 4, hooking its sampler's accept step to record which canvas positions commit, when, and at what confidence. Across a 686-prompt, six-regime probe suite we find that its decoding is neither parallel nor block-autoregressive: it follows a partial left-to-right commit bias whose apparent strength depends almost entirely on the granularity at which you look. Order is weak token by token and strengthens smoothly as the analysis is coarsened, so the model's "block size" turns out to be an artifact of the measuring ruler rather than the architecture. The model commits in large simultaneous batches, leaving much of the within-batch order genuinely undefined rather than merely unobserved. The behaviour is regime-dependent: structured JSON is committed in essentially arbitrary order, and a position's commit confidence tracks correctness on mathematical reasoning but carries no signal on factual recall. Commitment is aggressive, finishing in a short late burst well inside the step budget, while task accuracy matches the model's autoregressive Gemma-4 sibling. Beyond these findings, our central contribution is methodological: measuring decoding order honestly demands handling trailing-EOS padding, within-regime confounding, commit non-monotonicity, block-size sensitivity, and large commit-batch ties, each of which can otherwise manufacture a decoding-order result that is not really there.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 16 of its 18 references that are in the map. First posted on arXiv 12 Jun 2026.
Builds on
16 of 18 references.
Closest ancestors · 13
Same direction, newest first
- 2026Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment7 May
- 2026Simple Self-Conditioning Adaptation for Masked Diffusion Models28 Apr
- 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Models23 Mar
- 2026Unifying Masked Diffusion Models with Various Generation Orders and Beyond2 Feb
- 2026TABES: Trajectory-Aware Backward-on-Entropy Steering for Masked Diffusion Models30 Jan
- 2026Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space28 Jan
- 2026Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow22 Jan
- 2025dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learning24 Dec
- 2025Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty24 Dec
- 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Models7 Nov
- 2025A Survey on Diffusion Language Models14 Aug
- 2025Discrete Diffusion in Large Language and Multimodal Models: A Survey16 Jun
1 more
Other references · 3
- 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
- 2026CATTO: Balancing Preferences and Confidence in Language Modelscited by 4
- 2026Confidence Calibration in Large Language Modelscited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026DiffusionGemma Technical Reportcited by 11
- 2026How Transparent is DiffusionGemma?cited by 3
- 2026Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challengescited by 0
- 2026Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challenges4 Sep 2026
- 2026DiffusionGemma Technical Report31 Jul 2026
- 2026How Transparent is DiffusionGemma?18 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2025Diffusion Language Models Know the Answer Before Decoding
- 2026Commit Locally, Exit Globally: Coordinating Adaptive Sampling and Early Exit in Diffusion Language Models
- 2026Adaptation to Intrinsic Dependence in Diffusion Language Models
- 2026Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective
- 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Models
- 2026Don't Commit Alone: Joint Token Commitment in Diffusion Language Models
- 2026Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding
- 2026Deferred Commitment Decoding for Diffusion Language Models