mlsanityData to 7 Oct 2026

Limits of Confidence in Diffusion

arXiv 2609.20581 ↗First posted 17 Sep 2026cs.AIcited by 2

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Discrete diffusion, including remasking and uniform-state samplers, generate a sequence by writing multiple token positions per step, drawing each from a per-position distribution and choosing which positions to write from those same distributions. For domains of general interest (pixels, phonemes, or words) there are inherent dependencies between tokens. We show that a step matches the training distribution only when the positions it writes are conditionally independent given the tokens already fixed, that no product of per-position distributions can match a dependent group, and that per-position distributions do not determine whether a group is dependent: two joint distributions can have identical per-position marginals while differing in which combinations of values occur. On ScanAndAdd, a synthetic task whose joint distribution is available in closed form, we verify that every group of two or more undetermined positions a confidence ranking writes is dependent, and measure the generated distribution to be 29× the sampling-noise floor total variation while per-sample metrics are 1.0.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 15 of its 22 references that are in the map. First posted on arXiv 17 Sep 2026.

Builds on

15 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul

Other references · 14

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  5. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  6. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  7. 2017Non-Autoregressive Neural Machine Translationcited by 457
  8. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 181
  9. 2023Teaching Arithmetic to Small Transformerscited by 121
  10. 2022Teaching Algorithmic Reasoning via In-context Learningcited by 111
  11. 2026The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Modelscited by 16
  12. 2025Parallel Sampling from Masked Diffusion Models via Conditional Independence Testingcited by 13
2 more
  1. 2026Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspectivecited by 9
  2. 2026Dependency-Guided Parallel Decoding in Discrete Diffusion Language Modelscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Entropy Can Flow, or It Can Guide. Be Entropy. LEDFlow: Introducing Entropy-guided Generation Order into Uniform Discrete Flowcited by 0
  2. 2026On Trajectory-Aware Training for Masked Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Uniform Discrete Diffusion Models are Minimax Optimal for Estimating Distributions with Small Effective Support Size
  2. 2023The Emergence of Reproducibility and Generalizability in Diffusion Models
  3. 2025Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics
  4. 2026DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking
  5. 2026Let It Go or Learn to Self-Correct: Continuous Diffusion for Constrained Discrete Tasks
  6. 2026Generating from Discrete Distributions Using Diffusions: Insights from Random Constraint Satisfaction Problems
  7. 2026Adaptation to Intrinsic Dependence in Diffusion Language Models
  8. 2025Discrete Diffusion Models: Novel Analysis and New Sampler Guarantees