mlsanityData to 7 Oct 2026

Discrete Stochastic Localization for Non-autoregressive Generation

arXiv 2605.12836 ↗First posted 13 May 2026cs.LGcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Continuous diffusion is a natural framework for non-autoregressive generation but has generally lagged behind masked discrete diffusion models (MDMs) on discrete sequence generation. We argue that the bottleneck is not continuity itself, but a representation in which denoising depends on timestep-indexed noise regimes. We introduce Discrete Stochastic Localization (DSL), a continuous-state framework with unit-sphere token embeddings whose Bayes-optimal denoiser is invariant to the nominal signal-to-noise ratio (SNR) under the localization channel. One trained network then supports an entire family of per-token SNR paths, with endpoint masked-diffusion paths as a special case. Fine-tuning a pretrained MDLM checkpoint with DSL substantially improves distributional faithfulness (MAUVE) on OpenWebText across all step budgets from T=128 to T=1024, and the same checkpoint supports random-order autoregressive sampling, as well as a hybrid continuous-then-discrete sampler using as few as T=48 total steps -- without distillation or retraining.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 30 of its 37 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

30 of 37 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Is Noise Conditioning Necessary for Denoising Generative Models?18 Feb
  2. 2024Your Diffusion Model is Secretly a Noise Classifier and Benefits from Contrastive Training12 Jul
  3. 2023Information-Theoretic Diffusion7 Feb
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  6. 2020Denoising Diffusion Implicit Models6 Oct
  7. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Video Diffusion Modelscited by 2,280
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  4. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  5. 2025Large Language Diffusion Modelscited by 872
  6. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  7. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  9. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  10. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  11. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  12. 2020Hopfield Networks is All You Needcited by 461
11 more
  1. 2021Autoregressive Diffusion Modelscited by 208
  2. 2022Continuous diffusion for categorical datacited by 202
  3. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 181
  4. 2023Likelihood-Based Diffusion Language Modelscited by 147
  5. 2025The Diffusion Dualitycited by 124
  6. 2022Self-conditioned Embedding Diffusion for Text Generationcited by 114
  7. 2019Discrete Flows: Invertible Generative Models of Discrete Datacited by 68
  8. 2019Latent Normalizing Flows for Discrete Sequencescited by 66
  9. 2022Training and Inference on Any-Order Autoregressive Models the Right Waycited by 64
  10. 2025CANDI: Hybrid Discrete-Continuous Diffusion Modelscited by 31
  11. 2025Anchored Diffusion Language Modelcited by 17

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Discrete Stochastic Localization for Non-autoregressive Generation
  2. 2024Denoising Diffusions with Optimal Transport: Localization, Curvature, and Multi-Scale Complexity
  3. 2026Discrete Diffusion Models: A Unified Framework from Tokenization to Generation
  4. 2026DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs
  5. 2023Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces
  6. 2025Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusion
  7. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models
  8. 2025Almost Linear Convergence under Minimal Score Assumptions: Quantized Transition Diffusion