mlsanityData to 7 Oct 2026

Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models

arXiv 2601.07351 ↗First posted 12 Jan 2026cs.CL cs.AIcited by 6

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

Diffusion Language Models (DLMs) offer a promising alternative for language modeling by enabling parallel decoding through iterative refinement. However, most DLMs rely on hard binary masking and discrete token assignments, which hinder the revision of early decisions and underutilize intermediate probabilistic representations. In this paper, we propose EvoToken-DLM, a novel diffusion-based language modeling approach that replaces hard binary masks with evolving soft token distributions. EvoToken-DLM enables a progressive transition from masked states to discrete outputs, supporting revisable decoding. To effectively support this evolution, we introduce continuous trajectory supervision, which aligns training objectives with iterative probabilistic updates. Extensive experiments across multiple benchmarks show that EvoToken-DLM consistently achieves superior performance, outperforming strong diffusion-based and masked DLM baselines. Project webpage: https://aim-uofa.github.io/EvoTokenDLM.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 30 of its 34 references that are in the map. First posted on arXiv 12 Jan 2026.

Builds on

30 of 34 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  2. 2025Soft-Masked Diffusion Language Models20 Oct
  3. 2025Next Semantic Scale Prediction via Hierarchical Diffusion Language Models8 Oct
  4. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning6 Oct
  5. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner3 Oct
  6. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling1 Oct
  7. 2025Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models12 Aug
  8. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  9. 2025LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs17 Jun
  10. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  11. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  12. 2024Simple and Effective Masked Diffusion Language Models11 Jun
1 more
  1. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct

Other references · 17

  1. 2020Denoising Diffusion Implicit Modelscited by 9,655
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  5. 2023Let's Verify Step by Stepcited by 3,598
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  7. 2025s1: Simple test-time scalingcited by 1,311
  8. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  9. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  10. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  11. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  12. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMscited by 110
5 more
  1. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 109
  2. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Evictioncited by 64
  3. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decodingcited by 26
  4. 2025Text Generation Beyond Discrete Token Samplingcited by 21
  5. 2025Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibrationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 14
  2. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Modelscited by 7
  3. 2026DLLM Agent: See Farther, Run Fastercited by 5
  4. 2026Edit-Based Refinement for Parallel Masked Diffusion Language Modelscited by 1
  5. 2026Accelerating Diffusion Language Models via Structured Suffix Modelingcited by 0
  6. 2026Rethinking Soft Tokens for Parallel Decoding in Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Models
  2. 2025Sequential Diffusion Language Models
  3. 2026Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching
  4. 2025Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model
  5. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models
  6. 2026Beyond Masks: Efficient, Flexible Diffusion Language Models via Deletion-Insertion Processes
  7. 2025A Survey on Diffusion Language Models
  8. 2025Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models