mlsanityData to 14 Oct 2026

A Few More Tokens Ahead: Reinforcing Parallelism in Diffusion LLMs

arXiv 2610.18200 ↗First posted 13 Oct 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Parallel generation is widely regarded as an inherent advantage of diffusion large language models. Yet our fixed-quota evaluation of four representative models reveals severe degradation across mathematics and code, with all scores below 15% at 16 tokens per step. Token-level probes identify local predictive potential near the observed prefix, especially in mathematical text. We propose RePara, which reinforces short consecutive token blocks through group-relative policy optimization and reduces rollout and log-probability recomputation costs. On LLaDA and SDAR, ours improves GSM8K and MATH-500 accuracy over JustGRPO under reduced denoising budgets, with a 1.96x training-step speedup in our LLaDA/MATH setup.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

29 of 38 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  9. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  10. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  11. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  12. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
17 more
  1. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  2. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  3. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
  4. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  5. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  6. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  7. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  8. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  9. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
  10. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 194
  11. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 187
  12. 2026DFlash: Block Diffusion for Flash Speculative Decodingcited by 135
  13. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 129
  14. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 93
  15. 2025Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principlescited by 45
  16. 2025DiffuSpec: Unlocking Diffusion Language Models for Speculative Decodingcited by 42
  17. 2025Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspectivecited by 34

Built on by

Not cited yet.