Lost in Diffusion: Uncovering Hallucination Patterns and Failure Modes in Diffusion Large Language Models
Not on the map. Why?
Abstract (arXiv)
While Diffusion Large Language Models (dLLMs) have emerged as a promising non-autoregressive paradigm comparable to autoregressive (AR) models, their faithfulness, specifically regarding hallucination, remains largely underexplored. To bridge this gap, we present the first controlled comparative study to evaluate hallucination patterns in dLLMs. Our results demonstrate that current dLLMs exhibit a higher propensity for hallucination than AR counterparts controlled for architecture, scale, and pre-training weights. Furthermore, an analysis of inference-time compute reveals divergent dynamics: while quasi-autoregressive generation suffers from early saturation, non-sequential decoding unlocks potential for continuous refinement. Finally, we identify distinct failure modes unique to the diffusion process, including premature termination, incomplete denoising, and context intrusion. Our findings underscore that although dLLMs have narrowed the performance gap on general tasks, their distinct hallucination mechanisms pose a critical challenge to model reliability. Our code is available at https://github.com/ZeroLoss-Lab/Lost-in-Diffusion
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 18 of its 41 references that are in the map. First posted on arXiv 12 Apr 2026.
Builds on
18 of 41 references.
References · 18
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
- 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
- 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
- 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
- 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
- 2023How Language Model Hallucinations Can Snowballcited by 310
- 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
- 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 181
- 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
- 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
6 more
- 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 114
- 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoisecited by 79
- 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 42
- 2025Every Activation Boosted: Scaling General Reasoner to 1 Trillion Open Language Foundationcited by 32
- 2022What Makes Pre-trained Language Models Better Zero-shot Learners?cited by 23
- 2025Consultant Decoding: Yet Another Synergistic Mechanismcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Reliability Challenges in Diffusion Vision-Language Models
- 2026TDGNet: Hallucination Detection in Diffusion Language Models via Temporal Dynamic Graphs
- 2026DeMTS: Denoising Trajectories as Multivariate Time Series for Hallucination Detection in Diffusion Language Models
- 2025Towards Understanding Text Hallucination of Diffusion Models via Local Generation Bias
- 2025Dream 7B: Diffusion Large Language Models
- 2025A Survey on Diffusion Language Models
- 2026Unified Hallucination Fuzzing for Multimodal Large Language Models
- 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques