mlsanityData to 14 Oct 2026

Rethinking Soft Tokens for Parallel Decoding in Diffusion Language Models

arXiv 2609.37391 ↗First posted 29 Sep 2026cs.LGcited by 0

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

also in 2 more
Abstract (arXiv)

Diffusion language models (DLMs) enable parallel generation by predicting and committing multiple tokens at each denoising step, yet they can generate individually plausible but mutually inconsistent tokens. Recent work shows that soft tokens can mitigate this issue by representing uncertain positions with continuous embeddings built from the model's predictive distribution at the previous decoding step. However, although soft tokens are commonly understood as preserving predictive uncertainty, how soft-token feedback improves parallel decoding has not been systematically examined. In this paper, we investigate this question in frozen pretrained DLMs to examine soft-token feedback without the effects of additional training. To construct soft-token inputs in a training-free setting, we identify a geometric mismatch between conventional soft-token construction and the pretrained embedding space. Based on this observation, we propose a training-free, geometry-aware construction of soft tokens. Our analysis of soft-token feedback suggests that uncertainty preservation alone does not fully explain how it reshapes subsequent predictions. To better explain how soft-token feedback improves parallel decoding, we provide empirical evidence that it favors coherent token sequences. Across four pretrained DLMs and four math and code benchmarks, our method outperforms standard parallel decoding and a training-free Euclidean soft-token baseline. Code: https://github.com/kodaikawamura/rethinking-soft-tokens

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 34 of its 36 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

34 of 36 references.

References · 34

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2021Program Synthesis with Large Language Modelscited by 3,637
  4. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  5. 2019Root Mean Square Layer Normalizationcited by 1,480
  6. 2025Large Language Diffusion Modelscited by 912
  7. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  8. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  9. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  10. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  11. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  12. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
22 more
  1. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  2. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  3. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 225
  4. 2022Score-based Continuous-time Discrete Diffusion Modelscited by 169
  5. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 166
  6. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 148
  7. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 131
  8. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 117
  9. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMscited by 116
  10. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
  11. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chainscited by 85
  12. 2022From Denoising Diffusions to Denoising Markov Modelscited by 58
  13. 2024Discrete Copula Diffusioncited by 54
  14. 2025SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoningcited by 41
  15. 2023Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Timecited by 20
  16. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 18
  17. 2025Soft-Masked Diffusion Language Modelscited by 16
  18. 2026dLLM: Simple Diffusion Language Modelingcited by 14
  19. 2026Breaking the Factorization Barrier in Diffusion Language Modelscited by 11
  20. 2026Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Modelscited by 6
  21. 2022Uncertain Evidence in Probabilistic Models and Stochastic Simulatorscited by 5
  22. 2026Lost in Interpolation: Why Predictive Feedback Fails in Diffusion Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Mean-Field Parallel Decoding for Discrete Diffusion Language Models
  2. 2026Supportive Token Revealing for Fast Diffusion Language Model Decoding
  3. 2026Reliable Parallel Decoding in Masked Diffusion Language Models
  4. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models
  5. 2026Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation
  6. 2026Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?
  7. 2026Edit-Based Refinement for Parallel Masked Diffusion Language Models
  8. 2026LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detection