BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation

arXiv 2606.15893 ↗First posted 14 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Hallucinations remain a major obstacle to deploying large language models (LLMs) in knowledge-intensive settings, where generated responses must be faithfully grounded in provided evidence. Reinforcement learning (RL) is a promising direction for hallucination mitigation, but response-level faithfulness rewards suffer from a granularity mismatch: localized hallucinations can cause supported content to receive spurious penalties. Although recent work introduces fine-grained feedback such as claim-level verification and token-level rewards, unbalanced credit assignment can still induce length, verbosity, or optimization-noise biases. We propose BALTO, a Balanced Token-level Policy Optimization framework for hallucination mitigation. BALTO extracts checkable factual claims, verifies them against the reference context, and projects claim-level judgments to token-level labels. A balanced token-level credit assignment mechanism is introduced into the framework. This design redistributes probability mass from unsupported content toward faithful content, rather than suppressing the entire response. We systematically analyze the limitations of response-level rewards from a theoretical standpoint, and prove BALTO's advantages in training stability and optimization efficiency for hallucination mitigation. Experiments on ConFiQA, RAGTruth, and FinLLM-Eval show that BALTO achieves the highest faithfulness across all six model--benchmark settings and consistently outperforms existing post-training baselines in Q-Score, demonstrating a stronger faithfulness--informativeness trade-off.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 67 references that are in the map. First posted on arXiv 14 Jun 2026.

Builds on

43 of 67 references.

References · 43

2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
2025Qwen3 Technical Reportcited by 8,820
2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
2023Let's Verify Step by Stepcited by 3,789
2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
2022Survey of Hallucination in Natural Language Generationcited by 2,075
2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 431
2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 287

31 more

2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challengescited by 282
2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learningcited by 161
2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 150
2021Entity-level Factual Consistency of Abstractive Text Summarizationcited by 118
2024Reducing hallucination in structured outputs via Retrieval-Augmented Generationcited by 93
2025Medical Hallucinations in Foundation Models and Their Impact on Healthcarecited by 93
2024Dense Reward for Free in Reinforcement Learning from Human Feedbackcited by 78
2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
2025Stop Summation: Min-Form Credit Assignment Is All Process Reward Model Needs for Reasoningcited by 50
2025Reinforcement Learning Finetunes Small Subnetworks in Large Language Modelscited by 49
2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
2024TLCR: Token-Level Continuous Reward for Fine-grained Reinforcement Learning from Human Feedbackcited by 43
2023Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucinationcited by 41
2023Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logiccited by 35
2025CAPO: Towards Enhancing LLM Reasoning through Generative Credit Assignmentcited by 26
2024Mitigating Large Language Model Hallucination with Faithful Finetuningcited by 24
2025Learning to Reason for Factualitycited by 24
2025Benchmarking LLM Faithfulness in RAG with Evolving Leaderboardscited by 20
2025KnowRL: Exploring Knowledgeable Reinforcement Learning for Factualitycited by 20
2025KTAE: A Model-Free Algorithm to Key-Tokens Advantage Estimation in Mathematical Reasoningcited by 19
2025Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Modelscited by 19
2024Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuningcited by 16
2025Discriminative Policy Optimization for Token-Level Reward Modelscited by 16
2025FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Userscited by 15
2025Good Learners Think Their Thinking: Generative PRM Makes Large Reasoning Model More Efficient Math Learnercited by 11
2026Baichuan-M3: Modeling Clinical Inquiry for Reliable Medical Decision-Makingcited by 11
2024On-Policy Self-Alignment with Fine-grained Knowledge Feedback for Hallucination Mitigationcited by 9
2025Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Modelscited by 9
2025Knowledge-Level Consistency Reinforcement Learning: Dual-Fact Alignment for Long-Form Factualitycited by 4
2026Mitigating Hallucination in Financial Retrieval-Augmented Generation via Fine-Grained Knowledge Verificationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way