DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models
Not on the map. Why?
Abstract (arXiv)
The reliability of large language models remains a critical challenge, particularly due to their susceptibility to hallucinations and factual inaccuracies during text generation. Existing solutions either underutilize models' self-correction with preemptive strategies or use costly post-hoc verification. To further explore the potential of real-time self-verification and correction, we present Dynamic Self-Verify Decoding (DSVD), a novel decoding framework that enhances generation reliability through real-time hallucination detection and efficient error correction. DSVD integrates two key components: (1) parallel self-verification architecture for continuous quality assessment, (2) dynamic rollback mechanism for targeted error recovery. Extensive experiments across five benchmarks demonstrate DSVD's effectiveness, achieving significant improvement in truthfulness (Quesetion-Answering) and factual accuracy (FActScore). Results show the DSVD can be further incorporated with existing faithful decoding methods to achieve stronger performance. Our work establishes that real-time self-verification during generation offers a viable path toward more trustworthy language models without sacrificing practical deployability.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 25 of its 30 references that are in the map. First posted on arXiv 5 Mar 2025.
Builds on
25 of 30 references.
References · 25
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
- 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
- 2017Crowdsourcing Multiple Choice Science Questionscited by 742
- 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
- 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
13 more
- 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
- 2022Generating Sequences by Learning to Self-Correctcited by 271
- 2023Speculative Decoding with Big Little Decodercited by 163
- 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
- 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
- 2024TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Spacecited by 90
- 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
- 2023Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuningcited by 45
- 2024In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigationcited by 43
- 2023A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoningcited by 41
- 2024AMUSD: Asynchronous Multi-Device Speculative Decoding for LLM Accelerationcited by 13
- 2024SED: Self-Evaluation Decoding Enhances Large Language Models for Better Generationcited by 5
- 2025Efficient Real-time Refinement of Language Model Text Generationcited by 2
Built on by
Citing papers per year: 2025: 2.
2 papers.