# WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference

arXiv: https://arxiv.org/abs/2512.22737 · HTML: https://mlsanity.com/paper/2512.22737/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Placed in Discrete and text diffusion, one of 14 directions in Diffusion models.
- Builds on 31 of its 34 references that are in the map.
- First posted on arXiv 28 Dec 2025.

## Placed in

- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 30 of 31)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models](https://mlsanity.com/paper/2503.09573/) (2025)
- [Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data](https://mlsanity.com/paper/2406.03736/) (2024)
- [Scaling Diffusion Language Models via Adaptation from Autoregressive Models](https://mlsanity.com/paper/2410.17891/) (2024)
- [Scaling up Masked Diffusion Models on Text](https://mlsanity.com/paper/2410.18514/) (2024)
- [d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning](https://mlsanity.com/paper/2504.12216/) (2025)
- [LLaDA2.0: Scaling Up Diffusion Language Models to 100B](https://mlsanity.com/paper/2512.15745/) (2025)
- [SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation](https://mlsanity.com/paper/2510.06303/) (2025)
- [Dream-Coder 7B: An Open Diffusion Language Model for Code](https://mlsanity.com/paper/2509.01142/) (2025)
- [LLaDA-MoE: A Sparse MoE Diffusion Language Model](https://mlsanity.com/paper/2509.24389/) (2025)
- [dInfer: An Efficient Inference Framework for Diffusion Language Models](https://mlsanity.com/paper/2510.08666/) (2025)
- [Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed](https://mlsanity.com/paper/2512.14067/) (2025)
- [A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models](https://mlsanity.com/paper/2508.08712/) (2025)
- [d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation](https://mlsanity.com/paper/2509.21474/) (2025)
- [From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs](https://mlsanity.com/paper/2512.06776/) (2025)
- [Set Block Decoding is a Language Model Inference Accelerator](https://mlsanity.com/paper/2509.04185/) (2025)
- [Sequential Diffusion Language Models](https://mlsanity.com/paper/2509.24007/) (2025)

## Built on by (most cited first, 30 of 41)

- [LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding](https://mlsanity.com/paper/2605.27365/) (2026)
- [DMax: Aggressive Parallel Decoding for dLLMs](https://mlsanity.com/paper/2604.08302/) (2026)
- [Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Model](https://mlsanity.com/paper/2601.15892/) (2026)
- [DAWN: Dependency-Aware Fast Inference for Diffusion LLMs](https://mlsanity.com/paper/2602.06953/) (2026)
- [TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Acceleration](https://mlsanity.com/paper/2602.08404/) (2026)
- [Introspective Diffusion Language Models](https://mlsanity.com/paper/2604.11035/) (2026)
- [Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding](https://mlsanity.com/paper/2601.17917/) (2026)
- [ρ-textttEOS: Training-free Bidirectional Variable-Length Control for Masked Diffusion LLMs](https://mlsanity.com/paper/2601.22527/) (2026)
- [Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammars](https://mlsanity.com/paper/2602.00612/) (2026)
- [DSB: Dynamic Sliding Block Scheduling for Diffusion LLMs](https://mlsanity.com/paper/2602.05992/) (2026)
- [Improving Sampling for Masked Diffusion Models via Information Gain](https://mlsanity.com/paper/2602.18176/) (2026)
- [When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Models](https://mlsanity.com/paper/2604.23994/) (2026)
- [FLARE: Diffusion for Hybrid Language Model](https://mlsanity.com/paper/2606.01774/) (2026)
- [Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models](https://mlsanity.com/paper/2604.00375/) (2026)
- [ILRR: Inference-Time Steering Method for Masked Diffusion Language Models](https://mlsanity.com/paper/2601.21647/) (2026)
- [FOCUS: DLLMs Know How to Tame Their Compute Bound](https://mlsanity.com/paper/2601.23278/) (2026)
- [DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels](https://mlsanity.com/paper/2602.11715/) (2026)
- [From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Models](https://mlsanity.com/paper/2603.04592/) (2026)
- [Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Models](https://mlsanity.com/paper/2604.01622/) (2026)
- [Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding](https://mlsanity.com/paper/2607.01775/) (2026)
- [A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models](https://mlsanity.com/paper/2607.14522/) (2026)
- [GeoBlock: Inferring Block Granularity from Dependency Geometry in Diffusion Language Models](https://mlsanity.com/paper/2603.26675/) (2026)
- [Optimus: Elastic Decoding for Efficient Diffusion LLM Serving](https://mlsanity.com/paper/2605.24832/) (2026)
- [Triplet-Block Diffusion RWKV](https://mlsanity.com/paper/2605.25969/) (2026)
- [Enabling KV Caching of Shared Prefix for Diffusion Language Models](https://mlsanity.com/paper/2606.07571/) (2026)
- [VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Termination](https://mlsanity.com/paper/2606.17999/) (2026)
- [Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL](https://mlsanity.com/paper/2607.16204/) (2026)
- [AR-MAP: Are Autoregressive Large Language Models Implicit Teachers for Diffusion Large Language Models?](https://mlsanity.com/paper/2602.02178/) (2026)
- [Turning the TIDE: Cross-Architecture Distillation for Diffusion Large Language Models](https://mlsanity.com/paper/2604.26951/) (2026)
- [LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detection](https://mlsanity.com/paper/2605.10980/) (2026)

## Abstract (arXiv)

Autoregressive (AR) generation is the standard decoding paradigm for Large Language Models (LLMs), but its token-by-token nature limits parallelism at inference time. Diffusion Language Models (DLLMs) offer parallel decoding by recovering multiple masked tokens per step; however, in practice they often fail to translate this parallelism into deployment speed gains over optimized AR engines (e.g., vLLM). A key reason is that many DLLMs rely on bidirectional attention, which breaks standard prefix KV caching and forces repeated contextualization, undermining efficiency. We propose WeDLM, a diffusion decoding framework built entirely on standard causal attention to make parallel generation prefix-cache friendly. The core idea is to let each masked position condition on all currently observed tokens while keeping a strict causal mask, achieved by Topological Reordering that moves observed tokens to the physical prefix while preserving their logical positions. Building on this property, we introduce a streaming decoding procedure that continuously commits confident tokens into a growing left-to-right prefix and maintains a fixed parallel workload, avoiding the stop-and-wait behavior common in block diffusion methods. Experiments show that WeDLM preserves the quality of strong AR backbones while delivering substantial speedups, approaching 3x on challenging reasoning benchmarks and up to 10x in low-entropy generation regimes; critically, our comparisons are against AR baselines served by vLLM under matched deployment settings, demonstrating that diffusion-style decoding can outperform an optimized AR engine in practice.