Truth-Aware Decoding: A Program-Logic Approach to Factual Language Generation
Not on the map. Why?
Abstract (arXiv)
This paper introduces Truth-Aware Decoding (TAD), a verification-oriented decoding scheme that aligns neural language generation with knowledge bases. Situated in the tradition of probabilistic program semantics for sequence models, TAD augments modern instruction-tuned systems with a lattice of semantic guards that operate at decode time. Our contributions are fourfold: (i) a constraint-based semantics that renders oracle filtering as a program-logic judgment, (ii) a proof that greedy selection enjoys local likelihood dominance under sound and complete guards (Theorem 2.7), (iii) an entropy-style invariant that quantifies factual risk via knowledge-aware safe mass, and (iv) a multi-agent operational calculus with verified Lean artefacts to certify implementation behaviour. Numerical and algorithmic case studies confirm that the resulting guardrails reduce hallucinations without sacrificing throughput, yielding a pragmatic bridge between large-scale empirical models and formal verification.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 31 references that are in the map. First posted on arXiv 3 Oct 2025.
Builds on
25 of 31 references.
References · 25
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
13 more
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2019Language Models as Knowledge Bases?cited by 1,936
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
- 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
- 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
- 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
- 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
- 2019Neural Text Generation with Unlikelihood Trainingcited by 503
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026From Tokens to Steps: Verification-Aware Speculative Decoding for Efficient Multi-Step Reasoning
- 2026Neuro-Formal Verification: Agentic Language-Agnostic Formal Program Reasoning
- 2025A*-Decoding: Token-Efficient Inference Scaling
- 2026Thinking Before Constraining: A Unified Decoding Framework for Large Language Models
- 2026ConfSpec: Efficient Step-Level Speculative Reasoning via Confidence-Gated Verification
- 2026Beyond Solver Verdicts: Generative Reward Models for Autoformalization
- 2026Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States
- 2025Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match