mlsanityData to 14 Oct 2026

Uncertainty Quantification for Large Language Diffusion Models

arXiv 2605.14570 ↗First posted 14 May 2026cs.CLcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Large Language Diffusion Models (LLDMs) are emerging as an alternative to autoregressive models, offering faster inference through higher parallelism. Similar to autoregressive LLMs, they remain prone to hallucinations, making reliable uncertainty quantification (UQ) crucial for safe deployment. However, existing UQ methods are fundamentally misaligned with this new paradigm: they assume autoregressive factorization or use expensive repeated sampling, negating the efficiency of LLDMs. In this work, we present the first systematic study of UQ for LLDMs and propose lightweight, zero-shot uncertainty signals derived from the iterative denoising process, leveraging intermediate generations, token remasking dynamics, and denoising complexity. We further adapt a state-of-the-art UQ method to LLDMs by combining masked diffusion likelihoods with trajectory-based semantic dissimilarity. We prove that expected trajectory dissimilarity lower bounds the masked diffusion training objective, which motivates its usage as an uncertainty score. Comprehensive experiments across three tasks, eight datasets, and two models show that our method achieves a great cost-performance trade-off: it approaches the strongest sampling-based baselines while incurring up to 100x lower computational overhead. Our work demonstrates that LLDMs can deliver both fast inference and reliable hallucination detection simultaneously.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 39 of its 48 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

39 of 48 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Scaling Beyond Masked Diffusion Language Models16 Feb
  2. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  3. 2025Dream 7B: Diffusion Large Language Models21 Aug
  4. 2025The Diffusion Duality12 Jun
  5. 2025Large Language Diffusion Models14 Feb
  6. 2024Energy-Based Diffusion Language Models for Text Generation28 Oct
  7. 2024Simple and Effective Masked Diffusion Language Models11 Jun

Other references · 32

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  4. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  5. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  6. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  7. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  8. 2018CoQA: A Conversational Question Answering Challengecited by 683
  9. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  10. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
  11. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  12. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
20 more
  1. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 288
  2. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  3. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 211
  4. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 162
  5. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 159
  6. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
  7. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 142
  8. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 131
  9. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  10. 2024Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Modelscited by 86
  11. 2024LUQ: Long-text Uncertainty Quantification for LLMscited by 81
  12. 2023BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inferencecited by 35
  13. 2025Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistencycited by 30
  14. 2025Generative Uncertainty in Diffusion Modelscited by 23
  15. 2024Shedding Light on Large Generative Networks: Estimating Epistemic Uncertainty in Diffusion Modelscited by 22
  16. 2025Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Modelscited by 18
  17. 2025Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Headscited by 16
  18. 2026DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learningcited by 15
  19. 2024Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Modelscited by 9
  20. 2025TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Modelscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
  2. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs
  3. 2026LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
  4. 2026JUMP: Single-Pass Membership Inference on Fine-Tuned Diffusion Language Models
  5. 2025Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models
  6. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  7. 2026FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models
  8. 2026Diffusion Language Models: An Experimental Analysis