One-step lowest-variance selection in a Gaussian random-field model motivated by masked diffusion: Total correlation and a square root collision threshold
On the map: Diffusion models → Discrete and text diffusion
Abstract (arXiv)
Motivated by confidence-guided parallel unmasking in masked discrete diffusion, we study a single selection step in a stylized Gaussian random-field model. A locally dependent nonnegative score field represents position wise uncertainty, and the scheduler selects the K positions with the smallest scores. Dependence among the selected positions is measured through a distance-dependent Gaussian correlation model. This separation provides a tractable framework for quantifying how the geometry of low-score locations affects the dependence cost of factorized parallel decoding. We establish two complementary results. In a conservative sub-square-root regime, the conditional Gaussian total correlation of the selected block vanishes in probability. At the square-root scale, it remains non-negligible with positive asymptotic probability and admits a strictly positive expectation lower bound. Synthetic experiments support the predicted finite-size behavior. These results provide a rigorous stochastic-geometry baseline for understanding how budget size, score dependence, and spatial correlation jointly shape one-step confidence-based selection in masked discrete diffusion.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 12 of its 18 references that are in the map. First posted on arXiv 20 Jul 2026.
Builds on
12 of 18 references.
Closest ancestors · 8
Same direction, newest first
- 2025Learning Unmasking Policies for Diffusion Language Models9 Dec
- 2025Optimal Inference Schedules for Masked Diffusion Models6 Nov
- 2025Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policies7 Oct
- 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models27 May
- 2025Large Language Diffusion Models14 Feb
- 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
- 2021Autoregressive Diffusion Models5 Oct
Other references · 4
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
- 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
- 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
- 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Conditional Total Correlation and the Serial Depth of Adaptive Parallel Sampling
- 2026DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking
- 2026Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective
- 2026The data geometry of masking diffusion: Certified-optimal schedules via unmasking growth complexity
- 2026Diffusion Removes Langevin's Conditioning Dependence: A Sharp Gaussian Analysis
- 2026Latent Shadows: The Gaussian-Discrete Duality in Masked Diffusion
- 2025Bringing Stability to Diffusion: Decomposing and Reducing Variance of Training Masked Diffusion Models
- 2025Beyond Static Cutoffs: One-Shot Dynamic Thresholding for Diffusion Language Models