mlsanityData to 7 Oct 2026

Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models

arXiv 2508.09138 ↗First posted 12 Aug 2025cs.CL cs.AIcited by 33

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion large language models (dLLMs) generate text through iterative denoising, yet current decoding strategies discard rich intermediate predictions in favor of the final output. Our work here reveals a critical phenomenon, temporal oscillation, where correct answers often emerge in the middle process, but are overwritten in later denoising steps. To address this issue, we introduce two complementary methods that exploit temporal consistency: 1) Temporal Self-Consistency Voting, a training-free, test-time decoding strategy that aggregates predictions across denoising steps to select the most consistent output; and 2) a post-training method termed Temporal Consistency Reinforcement, which uses Temporal Semantic Entropy (TSE), a measure of semantic stability across intermediate predictions, as a reward signal to encourage stable generations. Empirical results across multiple benchmarks demonstrate the effectiveness of our approach. Using the negative TSE reward alone, we observe a remarkable average improvement of 24.7% on the Countdown dataset over an existing dLLM. Combined with the accuracy reward, we achieve absolute gains of 2.0% on GSM8K, 4.3% on MATH500, 6.6% on SVAMP, and 25.3% on Countdown, respectively. Our findings underscore the untapped potential of temporal dynamics in dLLMs and offer two simple yet effective tools to harness them.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 53 of its 59 references that are in the map. First posted on arXiv 12 Aug 2025.

Builds on

53 of 59 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  2. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  3. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  4. 2025Large Language Diffusion Models14 Feb
  5. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  6. 2024Scaling up Masked Diffusion Models on Text24 Oct
  7. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  9. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  10. 2023Likelihood-Based Diffusion Language Models30 May
  11. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct

Other references · 42

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  11. 2023Let's Verify Step by Stepcited by 3,598
  12. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
30 more
  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  3. 2022Video Diffusion Modelscited by 2,280
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  6. 2025s1: Simple test-time scalingcited by 1,311
  7. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  8. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  9. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  10. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  11. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  12. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
  13. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
  14. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  15. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  16. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  17. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 160
  18. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivizationcited by 124
  19. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 110
  20. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 107
  21. 2025Maximizing Confidence Alone Improves Reasoningcited by 97
  22. 2025SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimizationcited by 87
  23. 2023Dirichlet Diffusion Score Model for Biological Sequence Generationcited by 73
  24. 2024Fisher Flow Matching for Generative Modeling over Discrete Datacited by 68
  25. 2025VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuningcited by 61
  26. 2024Categorical Flow Matching on Statistical Manifoldscited by 43
  27. 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 38
  28. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMscited by 33
  29. 2025Temporal Sampling for Forgotten Reasoning in LLMscited by 12
  30. 2025Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theorycited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 19.

33 papers.

Sort:

By area:

  1. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  2. 2025Diffusion Language Models Know the Answer Before Decodingcited by 68
  3. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Modelscited by 32
  4. 2025AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Sizecited by 30
  5. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
  6. 2025DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Modelscited by 11
  7. 2025CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Creditcited by 11
  8. 2025Advancing Reasoning in Diffusion Language Models with Denoising Process Rewardscited by 9
  9. 2025Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Modelscited by 9
  10. 2025Test-Time Scaling in Diffusion LLMs via Hidden Semi-Autoregressive Expertscited by 8
  11. 2025Masks Can Be Distracting: On Context Comprehension in Diffusion Language Modelscited by 7
  12. 2026TDGNet: Hallucination Detection in Diffusion Language Models via Temporal Dynamic Graphscited by 7
21 more
  1. 2025Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wallcited by 6
  2. 2026Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Modelscited by 6
  3. 2026MetaState: Persistent Working Memory Enhances Reasoning in Discrete Diffusion Language Modelscited by 3
  4. 2025C2DLM: Causal Concept-Guided Diffusion Large Language Modelscited by 1
  5. 2026Tuning the Implicit Regularizer of Masked Diffusion Language Models: Enhancing Generalization via Insights from k-Paritycited by 1
  6. 2026SPA-Cache: Singular Proxies for Adaptive Caching in Diffusion Language Modelscited by 1
  7. 2026Stability-Weighted Decoding for Diffusion Language Modelscited by 1
  8. 2026The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Modelscited by 1
  9. 2026Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolationcited by 1
  10. 2026Where to Place the Query? Unveiling and Mitigating Positional Bias in In-Context Learning for Diffusion LLMs via Decoding Dynamicscited by 1
  11. 2026Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Modelscited by 1
  12. 2025STaRR: Spatial-Temporal Token-Dynamics-Aware Responsive Remasking for Diffusion Language Modelscited by 0
  13. 2026Progressive Refinement Regulation for Accelerating Diffusion Language Model Decodingcited by 0
  14. 2026Beyond Scattered Acceptance: Fast and Coherent Inference for DLMs via Longest Stable Prefixescited by 0
  15. 2026Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrastcited by 0
  16. 2026D2-Monitor: Dynamic Safety Monitoring for Diffusion LLMs via Hesitation Signalscited by 0
  17. 2026Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Modelscited by 0
  18. 2026Prefix-Denoising Consistency: Test-Time Verification for Diffusion Language Modelscited by 0
  19. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Samplingcited by 0
  20. 2026CaLR: Causal Latent Revision for Robust Diffusion Reasoningcited by 0
  21. 2026LOCKR: A Hidden-State Trajectory-Guided Planner for Detecting and Repairing Stable-but-Wrong Lock-In in Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Measuring Temporal Linguistic Emergence in Diffusion Language Models
  2. 2026Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Models
  3. 2025A Survey on Diffusion Language Models
  4. 2026Diffusion Language Models: An Experimental Analysis
  5. 2026Subliminal Clocks: Latent Time Modelling in Diffusion Language Models
  6. 2026LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
  7. 2026Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Models
  8. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think