mlsanityData to 7 Oct 2026

Introspective Diffusion Language Models

arXiv 2604.11035 ↗First posted 13 Apr 2026cs.AIcited by 6

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion language models promise parallel generation, yet still lag behind autoregressive (AR) models in quality. We stem this gap to a failure of introspective consistency: AR models agree with their own generations, while DLMs often do not. We define the introspective acceptance rate, which measures whether a model accepts its previously generated tokens. This reveals why AR training has a structural advantage: causal masking and logit shifting implicitly enforce introspective consistency. Motivated by this observation, we introduce Introspective Diffusion Language Model (I-DLM), a paradigm that retains diffusion-style parallel decoding while inheriting the introspective consistency of AR training. I-DLM uses a novel introspective strided decoding (ISD) algorithm, which enables the model to verify previously generated tokens while advancing new ones in the same forward pass. From a systems standpoint, we build I-DLM inference engine on AR-inherited optimizations and further customize it with a stationary-batch scheduler. To the best of our knowledge, I-DLM is the first DLM to match the quality of its same-scale AR counterpart while outperforming prior DLMs in both model quality and practical serving efficiency across 15 benchmarks. It reaches 69.6 on AIME-24 and 45.7 on LiveCodeBench-v6, exceeding LLaDA-2.1-mini (16B) by more than 26 and 15 points, respectively. Beyond quality, I-DLM is designed for the growing demand of large-concurrency serving, delivering about 3x higher throughput than prior state-of-the-art DLMs.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 41 of its 42 references that are in the map. First posted on arXiv 13 Apr 2026.

Builds on

41 of 42 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editing9 Feb
  2. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference28 Dec
  3. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed16 Dec
  4. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  5. 2025From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs7 Dec
  6. 2025TiDAR: Think in Diffusion, Talk in Autoregression12 Nov
  7. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  8. 2025Dream 7B: Diffusion Large Language Models21 Aug
  9. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  10. 2025Large Language Diffusion Models14 Feb
  11. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Time28 Oct
  12. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
2 more
  1. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  2. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 27

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2021Program Synthesis with Large Language Modelscited by 3,509
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  10. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  12. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
15 more
  1. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  2. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  3. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  4. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
  5. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  6. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 344
  7. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
  8. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  9. 2024MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmarkcited by 103
  10. 2024CLLMs: Consistency Large Language Modelscited by 71
  11. 2025Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potentialcited by 46
  12. 2025Set Block Decoding is a Language Model Inference Acceleratorcited by 22
  13. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Modelscited by 19
  14. 2025Fast and Accurate Causal Parallel Decoding using Jacobi Forcingcited by 11
  15. 2026LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Trees from Marginals: Autoregressive drafting with factorized priorscited by 2
  2. 2026The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Modelscited by 1
  3. 2026Re-evaluating Confidence Remasking in Masked Diffusion Language Modelscited by 1
  4. 2026Unlocking Lossless Speedups in LLMs via Discrete Diffusioncited by 1
  5. 2026Subliminal Clocks: Latent Time Modelling in Diffusion Language Modelscited by 0
  6. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Diffusion Language Models
  2. 2026Locally Coherent Parallel Decoding in Diffusion Language Models
  3. 2025How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practices
  4. 2026Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?
  5. 2025Diffusion Language Models are Super Data Learners
  6. 2025Diffusion Language Models Know the Answer Before Decoding
  7. 2026Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment
  8. 2026Rethinking the Generation Order of Block Diffusion Language Models