mlsanityData to 7 Oct 2026

LoPA: Scaling dLLM Inference via Lookahead Parallel Decoding

arXiv 2512.16229 ↗First posted 18 Dec 2025cs.CLcited by 17

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Diffusion Large Language Models (dLLMs) have demonstrated significant potential for high-speed inference. However, current confidence-driven decoding strategies are constrained by limited parallelism, typically achieving only 1--3 tokens per forward pass (TPF). In this work, we identify that the degree of parallelism during dLLM inference is highly sensitive to the Token Filling Order (TFO). Then, we introduce Lookahead PArallel Decoding LoPA, a training-free, plug-and-play algorithm, to identify a superior TFO and hence accelerate inference. LoPA concurrently explores distinct candidate TFOs via parallel branches, and selects the one with the highest potential for future parallelism based on branch confidence. We apply LoPA to the state-of-the-art D2F model and observe a substantial enhancement in decoding efficiency. Notably, LoPA increases the TPF of D2F-Dream to 10.1 on the GSM8K while maintaining performance superior to the Dream baseline. Furthermore, to facilitate this unprecedented degree of parallelism, we develop a specialized multi-device inference system featuring Branch Parallelism (BP), which achieves a single-sample throughput of 1073.9 tokens per second under multi-GPU deployment. The code is available at https://github.com/zhijie-group/LoPA.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 28 of its 34 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

28 of 34 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit7 Oct
  2. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models30 Sep
  3. 2025dParallel: Learnable Parallel Decoding for dLLMs30 Sep
  4. 2025Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphs22 Sep
  5. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  6. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decoding31 May
  7. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May
  8. 2025dKV-Cache: The Cache for Diffusion Language Models21 May

Other references · 20

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  7. 2021Program Synthesis with Large Language Modelscited by 3,509
  8. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  9. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  10. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  11. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  12. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
8 more
  1. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
  2. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  3. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 295
  4. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
  5. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  6. 2023Accelerating LLM Inference with Staged Speculative Decodingcited by 151
  7. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  8. 2025Diffusion Language Models Know the Answer Before Decodingcited by 68

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 17.

17 papers.

Sort:

By area:

  1. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
  2. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 14
  3. 2026Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Modelcited by 12
  4. 2026TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decodingcited by 7
  5. 2026LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learningcited by 6
  6. 2026Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Modelscited by 5
  7. 2026TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLMcited by 2
  8. 2026Multi-Block Diffusion Language Modelscited by 2
  9. 2026TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decodingcited by 2
  10. 2026GeoBlock: Inferring Block Granularity from Dependency Geometry in Diffusion Language Modelscited by 1
  11. 2026Where to Place the Query? Unveiling and Mitigating Positional Bias in In-Context Learning for Diffusion LLMs via Decoding Dynamicscited by 1
  12. 2026Ripple-Pivot Search: Active Parallel Decoding for Diffusion Large Language Modelscited by 1
5 more
  1. 2026LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detectioncited by 0
  2. 2026PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decodingcited by 0
  3. 2026Faster Block-Diffusion Serving with Distribution-Free Risk Guaranteescited by 0
  4. 2026Reliable Parallel Decoding in Masked Diffusion Language Modelscited by 0
  5. 2026Blackboard Intelligence Can Surpass Autoregressive on Globally Constrained Problemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
  2. 2026Fast-dLLM++: Fr'echet Profile Decoding for Faster Diffusion LLM Inference
  3. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
  4. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs
  5. 2025Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models
  6. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs
  7. 2026DAWN: Dependency-Aware Fast Inference for Diffusion LLMs
  8. 2025ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMs