mlsanityData to 7 Oct 2026

Token-Guard: Towards Token-Level Hallucination Control via Self-Checking Decoding

arXiv 2601.21969 ↗First posted 29 Jan 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) often hallucinate, generating content inconsistent with the input. Retrieval-Augmented Generation (RAG) and Reinforcement Learning with Human Feedback (RLHF) can mitigate hallucinations but require resource-intensive retrieval or large-scale fine-tuning. Decoding-based methods are lighter yet lack explicit hallucination control. To address this, we present Token-Guard, a token-level hallucination control method based on self-checking decoding. Token-Guard performs internal verification at each reasoning step to detect hallucinated tokens before they propagate. Candidate fragments are further evaluated in a latent space with explicit hallucination risk scoring, while iterative pruning and regeneration dynamically correct detected errors. Experiments on HALU datasets show Token-Guard substantially reduces hallucinations and improves generation accuracy, offering a scalable, modular solution for reliable LLM outputs. Our code is publicly available.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 44 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

36 of 44 references.

References · 36

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2020Learning to summarize from human feedbackcited by 2,562
  8. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  9. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  10. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  11. 2023Large Language Modelscited by 774
  12. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
24 more
  1. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  2. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  3. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  4. 2023How Language Model Hallucinations Can Snowballcited by 310
  5. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
  6. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  7. 2023Detecting and Preventing Hallucinations in Large Vision Language Modelscited by 254
  8. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 195
  9. 2024Token-level Direct Preference Optimizationcited by 135
  10. 2023Branch-Solve-Merge Improves Large Language Model Evaluation and Generationcited by 104
  11. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
  12. 2023Towards Mitigating Hallucination in Large Language Models via Self-Reflectioncited by 81
  13. 2024Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Basescited by 76
  14. 2025HyperGraphRAG: Retrieval-Augmented Generation via Hypergraph-Structured Knowledge Representationcited by 68
  15. 2024Lynx: An Open Source Hallucination Evaluation Modelcited by 55
  16. 2025Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learningcited by 50
  17. 2023KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detectioncited by 43
  18. 2024IterGen: Iterative Semantic-aware Structured LLM Generation with Backtrackingcited by 28
  19. 2024Non-myopic Generation of Language Models for Reasoning and Planningcited by 25
  20. 2025TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoningcited by 24
  21. 2025KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Searchcited by 21
  22. 2025φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitationcited by 12
  23. 2025Reflection-Window Decoding: Text Generation with Selective Refinementcited by 7
  24. 2025Let's Revise Step-by-Step: A Unified Local Search Framework for Code Generation with LLMscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Monitoring Decoding: Mitigating Hallucination via Evaluating the Factuality of Partial Response during Generation
  2. 2026Scalable Token-Level Hallucination Detection in Large Language Models
  3. 2025Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling
  4. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models
  5. 2026RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration
  6. 2025AutoRAG-LoRA: Hallucination-Triggered Knowledge Retuning via Lightweight Adapters
  7. 2026MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination
  8. 2025Theoretical Foundations and Mitigation of Hallucination in Large Language Models