mlsanityData to 14 Oct 2026

Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief States

arXiv 2510.11052 ↗First posted 13 Oct 2025cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Autoregressive (AR) models remain the standard for natural language generation but still suffer from high latency due to strictly sequential decoding. Recent diffusion-inspired approaches, such as LlaDA and Dream, mitigate this by generating in parallel, yet they suffer from two core limitations: information loss, as predictive distributions for non-finalized tokens are discarded at each step, and premature commitment, where local decisions are made without sufficient global coordination. We introduce Latent Refinement Decoding (LRD), a two-stage framework with Latent Refinement and a Predictive Feedback Loop. The first stage maintains masked positions as distributional mixtures of predicted tokens and the mask embedding, allowing the model to establish more globally consistent beliefs. The second stage progressively finalizes confident tokens while retaining uncertain ones for iterative feedback. KL-divergence dynamics provide a principled and reliable criterion for convergence and early stopping. Experiments across coding (HumanEval +6.3, MBPP +2.6) and reasoning (GSM8K +2.9, MATH500 +3.8) show that LRD improves accuracy while delivering speedups of up to 10.6x, making it a strong and versatile alternative for parallel sequence generation.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 40 of its 43 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

40 of 43 references.

References · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  8. 2025Qwen3 Technical Reportcited by 8,820
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  10. 2023Let's Verify Step by Stepcited by 3,789
  11. 2021Program Synthesis with Large Language Modelscited by 3,637
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
28 more
  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  2. 2025Large Language Diffusion Modelscited by 912
  3. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  4. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  5. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  6. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  7. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 381
  8. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 324
  9. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 274
  10. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generationcited by 257
  11. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 235
  12. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  13. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  14. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 143
  15. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 131
  16. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 117
  17. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
  18. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  19. 2025Diffusion Language Models Know the Answer Before Decodingcited by 70
  20. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Evictioncited by 65
  21. 2023Guiding Language Model Reasoning with Planning Tokenscited by 58
  22. 2021Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networkscited by 39
  23. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Modelscited by 34
  24. 2025Advancing Sequential Numerical Prediction in Autoregressive Modelscited by 26
  25. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Modelscited by 23
  26. 2025Pay Attention to Attention Distribution: A New Local Lipschitz Bound for Transformerscited by 11
  27. 2024Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelinescited by 6
  28. 2024SpecFormer: Guarding Vision Transformer Robustness via Maximum Singular Value Penalizationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Accelerationcited by 8
  2. 2026Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decodingcited by 7
  3. 2026Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decodingcited by 6
  4. 2026Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Modelscited by 3
  5. 2026MetaState: Persistent Working Memory Enhances Reasoning in Discrete Diffusion Language Modelscited by 3
  6. 2026Edit-Based Refinement for Parallel Masked Diffusion Language Modelscited by 2
  7. 2026Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokenscited by 1
  8. 2026Progressive Refinement Regulation for Accelerating Diffusion Language Model Decodingcited by 0
  9. 2026Accelerating Diffusion Language Models via Structured Suffix Modelingcited by 0