mlsanityData to 14 Oct 2026

Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model

arXiv 2510.18165 ↗First posted 20 Oct 2025cs.AI cs.CL cs.LGcited by 13

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion language models (DLMs) are emerging as a compelling alternative to the dominant autoregressive paradigm, offering inherent advantages in parallel generation and bidirectional context modeling. However, for the tasks with strict structural constraints such as code generation, DLMs face a critical trade-off between inference speed and output quality, where accelerating generation by reducing sampling steps often leads to catastrophic performance collapse. We find that the fundamental reasons are: 1) the generation difficulty is non-uniform in the structured sequence decoding steps, making DLM's static acceleration strategy suboptimal; 2) the context of tokens generated by DLM evolves continuously, causing early high-confidence predictions to turn into irreversible errors. In this paper, we introduce efficient Sampling with Adaptive acceleration and Backtracking Enhanced Remasking (i.e., Saber), a novel training-free sampling algorithm for DLMs that first achieves both better inference speed and output quality in code generation. Saber dynamically adjusts the number of tokens unmasked per step based on the model's evolving confidence, and utilizes a backtracking mechanism to revert tokens whose confidence drops as new context emerges, with its effectiveness supported by theoretical analysis. Extensive experiments on multiple mainstream code generation benchmarks show that Saber boosts Pass@1 accuracy by an average of 1.9% over mainstream DLM sampling methods, while achieving an average 251.4% inference speedup. By leveraging the inherent advantages of DLMs, our work significantly narrows the performance gap with autoregressive models in code generation.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 43 of its 55 references that are in the map. First posted on arXiv 20 Oct 2025.

Builds on

43 of 55 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Dream-Coder 7B: An Open Diffusion Language Model for Code1 Sep
  2. 2025Dream 7B: Diffusion Large Language Models21 Aug
  3. 2025A Survey on Diffusion Language Models14 Aug
  4. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models12 Aug
  5. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  6. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models23 Jun
  7. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  8. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding22 May
  9. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  10. 2025Large Language Diffusion Models14 Feb
  11. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  12. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct
2 more
  1. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  2. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov

Other references · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  7. 2021Program Synthesis with Large Language Modelscited by 3,637
  8. 2022Emergent Abilities of Large Language Modelscited by 2,512
  9. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  10. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  12. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
17 more
  1. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  2. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  3. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  4. 2023StarCoder: may the source be with you!cited by 799
  5. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  6. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 361
  7. 2023Self-collaboration Code Generation via ChatGPTcited by 258
  8. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 235
  9. 2023Self-planning Code Generation with Large Language Modelscited by 124
  10. 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 122
  11. 2025A Survey on Code Generation with LLM-based Agentscited by 106
  12. 2016Latent Predictor Networks for Code Generationcited by 73
  13. 2023CodeScore: Evaluating Code Generation by Learning Code Executioncited by 67
  14. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 46
  15. 2025RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimizationcited by 35
  16. 2024ROCODE: Integrating Backtracking Mechanism and Program Analysis in Large Language Models for Code Generationcited by 13
  17. 2025Rethinking Repetition Problems of LLMs in Code Generationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 10.

13 papers.

Sort:

By area:

  1. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 30
  2. 2026Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decodingcited by 6
  3. 2026Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Modelscited by 4
  4. 2026Adaptive Test-Time Compute Allocation for Block Diffusion Language Models in Complex Reasoningcited by 2
  5. 2026Tuning the Implicit Regularizer of Masked Diffusion Language Models: Enhancing Generalization via Insights from k-Paritycited by 1
  6. 2026Reversible Diffusion Decoding for Diffusion Language Modelscited by 1
  7. 2026The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Modelscited by 1
  8. 2026Re-evaluating Confidence Remasking in Masked Diffusion Language Modelscited by 1
  9. 2026Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokenscited by 1
  10. 2026Beyond Block Boundaries: Multi-Block Editing for Diffusion Large Language Modelscited by 1
  11. 2025Context-Aware Initialization for Reducing Generative Path Length in Diffusion Language Modelscited by 0
  12. 2025STaRR: Spatial-Temporal Token-Dynamics-Aware Responsive Remasking for Diffusion Language Modelscited by 0
1 more
  1. 2026Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models
  2. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Models
  3. 2025Diffusion Language Models are Provably Optimal Parallel Samplers
  4. 2026Rethinking the Generation Order of Block Diffusion Language Models
  5. 2026CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models
  6. 2026A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models
  7. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques
  8. 2026CORE: Context-Robust Remasking for Diffusion Language Models