mlsanityData to 7 Oct 2026

Don't Commit Alone: Joint Token Commitment in Diffusion Language Models

arXiv 2607.04469 ↗First posted 5 Jul 2026cs.CLcited by 2

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion language models (dLLMs) commit multiple tokens per denoising step by decoding each selected position independently from a shared context. When these positions are dependent, this factorization introduces an error captured by conditional total correlation, which confidence-based selection cannot infer from marginal probabilities alone. We propose CoCommit, a marker-gated coordination pass that delays commitment. After the usual bundle selection, a learned marker identifies the commit set, and the backbone's last n layers are re-applied to coordinate the marked positions before greedy argmax writes the tokens. This approximates joint-mode decoding while reusing existing weights, requiring only one partial forward pass and no auxiliary model. On LLaDA 2.1 with LoRA adapters and greedy inference, joint commitment improves five of the seven evaluated benchmarks over the released factorized decoder. The largest gains occur on code and reasoning tasks, while the remaining tasks are near parity.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 32 of its 36 references that are in the map. First posted on arXiv 5 Jul 2026.

Builds on

32 of 36 references.

Closest ancestors · 18

Same direction, newest first

  1. 2026Improved Large Language Diffusion Models24 Jun
  2. 2026Self-Generated Error Training for Token Editing in Diffusion Language Models15 Jun
  3. 2026ELF: Embedded Language Flows11 May
  4. 2026Continuous Latent Diffusion Language Model7 May
  5. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Models12 Feb
  6. 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editing9 Feb
  7. 2026CORE: Context-Robust Remasking for Diffusion Language Models4 Feb
  8. 2025Fine-Tuning Masked Diffusion for Provable Self-Correction1 Oct
  9. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models28 Sep
  10. 2025Dream 7B: Diffusion Large Language Models21 Aug
  11. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Models18 Aug
  12. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
6 more
  1. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  2. 2024Discrete Copula Diffusion2 Oct
  3. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  4. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  5. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  6. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 14

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  6. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  7. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  8. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  9. 2018Universal Transformerscited by 613
  10. 2019Deep Equilibrium Modelscited by 603
  11. 2017Non-Autoregressive Neural Machine Translationcited by 457
  12. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
2 more
  1. 2023CMATH: Can Your Language Model Pass Chinese Elementary School Math Test?cited by 89
  2. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026From Truncation to Commitment: Persistent Context in Uniform Discrete Diffusioncited by 1
  2. 2026A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Deferred Commitment Decoding for Diffusion Language Models
  2. 2026TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding
  3. 2026The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models
  4. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Models
  5. 2026Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
  6. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Models
  7. 2026Fast-dLLM++: Fr'echet Profile Decoding for Faster Diffusion LLM Inference
  8. 2026LESS Is More: Mutual-Stability Sampling for Diffusion Language Models