Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens
Not on the map. Why?
Abstract (arXiv)
Diffusion Large Language Models (dLLMs) offer a promising avenue for parallel generation but face a trade-off between decoding speed and quality. While revocable decoding strategies attempt to mitigate errors by verifying and remasking tokens, they typically operate within a mixed-quality context. This leads to two critical failures: textitError Propagation, where new tokens absorb toxic information from erroneous context, and textitLocal Error Reinforcement, where errors mutually reinforce each other to evade detection. To alleviate these challenges, we propose ASRD (Anchor Supervised Revocable Decoding), a training-free framework that operates within the embedding space. ASRD explicitly decouples the decoding context into trusted textitAnchor Tokens, which are identified via temporal consistency, and uncertain candidates. Leveraging a dynamic Anchor Tokens Cache, we introduce two complementary mechanisms: (1) Anchor-Guided Generation, which injects entropy-weighted anchor signals into masked positions to implicitly rectify attention toward the reliable global skeleton; and (2) Anchor-Perturbed Verification, which applies orthogonal perturbations to uncertain candidate tokens, destabilizing and remasking errors driven by fragile local consensus. Extensive experiments on math and coding benchmarks demonstrate that ASRD outperforms recent remasking baselines, achieving accuracy improvements of up to 6.4% while accelerating inference throughput by up to 7.2×.The code is available at https://github.com/preordinary/ASRD.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 37 of its 57 references that are in the map. First posted on arXiv 15 Jun 2026.
Builds on
37 of 57 references.
References · 37
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023Let's Verify Step by Stepcited by 3,789
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
- 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
- 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
- 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
- 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
- 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
25 more
- 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
- 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 381
- 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
- 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
- 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 189
- 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 187
- 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
- 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 166
- 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 129
- 2022Self-conditioned Embedding Diffusion for Text Generationcited by 117
- 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 116
- 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
- 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
- 2025Diffusion Language Models Know the Answer Before Decodingcited by 70
- 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusioncited by 65
- 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Evictioncited by 65
- 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 46
- 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 43
- 2025Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Modelcited by 13
- 2025Parallel Sampling from Masked Diffusion Models via Conditional Independence Testingcited by 13
- 2025Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief Statescited by 9
- 2025Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Modelscited by 9
- 2026Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decodingcited by 7
- 2026Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decodingcited by 6
- 2025Rethinking Reasoning with MDLMs: Early Exits, Post-hoc Reasoning, and Beyondcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Affix Cache for Diffusion Large Language Modelscited by 0
- 2026Affix Cache for Diffusion Large Language Models26 Jun 2026