# Neither Parallel Nor Sequential: How DiffusionGemma Actually Commits Tokens

arXiv: https://arxiv.org/abs/2606.14620 · HTML: https://mlsanity.com/paper/2606.14620/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Discrete and text diffusion, one of 14 directions in Diffusion models.
- Builds on 16 of its 18 references that are in the map.
- First posted on arXiv 12 Jun 2026.

## Placed in

- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 16 of 16)

- [Accelerating Diffusion LLMs via Adaptive Parallel Decoding](https://mlsanity.com/paper/2506.00413/) (2025)
- [A Survey on Diffusion Language Models](https://mlsanity.com/paper/2508.10875/) (2025)
- [Discrete Diffusion in Large Language and Multimodal Models: A Survey](https://mlsanity.com/paper/2506.13759/) (2025)
- [KLASS: KL-Guided Fast Inference in Masked Diffusion Models](https://mlsanity.com/paper/2511.05664/) (2025)
- [Confidence-Based Decoding is Provably Efficient for Diffusion Language Models](https://mlsanity.com/paper/2603.22248/) (2026)
- [Flexible-length Text Infilling for Discrete Diffusion Models](https://mlsanity.com/paper/2506.13579/) (2025)
- [Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow](https://mlsanity.com/paper/2601.15593/) (2026)
- [dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learning](https://mlsanity.com/paper/2512.21446/) (2025)
- [Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space](https://mlsanity.com/paper/2601.20339/) (2026)
- [Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty](https://mlsanity.com/paper/2512.21336/) (2025)
- [Unifying Masked Diffusion Models with Various Generation Orders and Beyond](https://mlsanity.com/paper/2602.02112/) (2026)
- [Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment](https://mlsanity.com/paper/2605.06885/) (2026)
- [CATTO: Balancing Preferences and Confidence in Language Models](https://mlsanity.com/paper/2601.23096/) (2026)
- [Simple Self-Conditioning Adaptation for Masked Diffusion Models](https://mlsanity.com/paper/2604.26985/) (2026)
- [TABES: Trajectory-Aware Backward-on-Entropy Steering for Masked Diffusion Models](https://mlsanity.com/paper/2602.00250/) (2026)
- [Confidence Calibration in Large Language Models](https://mlsanity.com/paper/2605.23909/) (2026)

## Built on by (most cited first, 3 of 3)

- [DiffusionGemma Technical Report](https://mlsanity.com/paper/2608.00146/) (2026)
- [How Transparent is DiffusionGemma?](https://mlsanity.com/paper/2606.20560/) (2026)
- [Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challenges](https://mlsanity.com/paper/2609.04778/) (2026)

## Abstract (arXiv)

Open diffusion language models are marketed as parallel, non-autoregressive decoders, yet the order in which a shipped checkpoint actually commits its tokens is almost never measured. We instrument DiffusionGemma 26B, a masked discrete-diffusion mixture-of-experts model built on Gemma 4, hooking its sampler's accept step to record which canvas positions commit, when, and at what confidence. Across a 686-prompt, six-regime probe suite we find that its decoding is neither parallel nor block-autoregressive: it follows a partial left-to-right commit bias whose apparent strength depends almost entirely on the granularity at which you look. Order is weak token by token and strengthens smoothly as the analysis is coarsened, so the model's "block size" turns out to be an artifact of the measuring ruler rather than the architecture. The model commits in large simultaneous batches, leaving much of the within-batch order genuinely undefined rather than merely unobserved. The behaviour is regime-dependent: structured JSON is committed in essentially arbitrary order, and a position's commit confidence tracks correctness on mathematical reasoning but carries no signal on factual recall. Commitment is aggressive, finishing in a short late burst well inside the step budget, while task accuracy matches the model's autoregressive Gemma-4 sibling. Beyond these findings, our central contribution is methodological: measuring decoding order honestly demands handling trailing-EOS padding, within-regime confounding, commit non-monotonicity, block-size sensitivity, and large commit-batch ties, each of which can otherwise manufacture a decoding-order result that is not really there.