mlsanityData to 7 Oct 2026

Think While You Write: Hypothesis Verification Promotes Faithful Knowledge-to-Text Generation

arXiv 2311.09467 ↗First posted 16 Nov 2023cs.CL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Knowledge-to-text generators often struggle to faithfully generate descriptions for the input facts: they may produce hallucinations that contradict the input, or describe facts not present in the input. To reduce hallucinations, we propose a decoding-only method, TWEAK (Think While Effectively Articulating Knowledge), which can be integrated with any generator without retraining. TWEAK treats the generated sequences at each decoding step and its future sequences as hypotheses, and ranks each generation candidate based on the extent to which their hypotheses are supported by the input facts using a Hypothesis Verification Model (HVM). We first demonstrate the effectiveness of TWEAK by using a Natural Language Inference (NLI) model as the HVM and report improved faithfulness with a minimal impact on the quality. We then replace the NLI model with a task-specific HVM trained with a first-of-a-kind dataset, FATE (Fact-Aligned Textual Entailment), which pairs input facts with their original and perturbed descriptions. We test TWEAK with two generators, and the best TWEAK variants improve on average for the two models by 2.24/7.17 points in faithfulness (FactKB) in in/out-of-distribution evaluations, respectively, and with only a 0.14/0.32-point decline in quality (BERTScore).

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 41 of its 53 references that are in the map. First posted on arXiv 16 Nov 2023.

Builds on

41 of 53 references.

References · 41

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  6. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  7. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  8. 2023How Language Model Hallucinations Can Snowballcited by 310
  9. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  10. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  11. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 153
  12. 2019Generalization in Generation: A closer look at Exposure Biascited by 123
29 more
  1. 2019Text Generation from Knowledge Graphs with Graph Transformerscited by 122
  2. 2020Investigating Pretrained Language Models for Graph-to-Text Generationcited by 117
  3. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  4. 2018Data-to-Text Generation with Content Selection and Planningcited by 106
  5. 2022TableFormer: Robust Transformer Modeling for Table-Text Encodingcited by 90
  6. 2023FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledgecited by 55
  7. 2023Detecting and Mitigating Hallucinations in Multilingual Summarisationcited by 48
  8. 2022RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Groundingcited by 47
  9. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  10. 2021UniRE: A Unified Label Space for Entity Relation Extractioncited by 38
  11. 2021Data-to-text Generation with Macro Planningcited by 34
  12. 2022Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarizationcited by 34
  13. 2022Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learningcited by 32
  14. 2021Commonsense-Focused Dialogues for Response Generation: An Empirical Studycited by 30
  15. 2019Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learningcited by 29
  16. 2021Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Modelscited by 29
  17. 2023Elastic Weight Removal for Faithful and Abstractive Dialogue Generationcited by 29
  18. 2018Deep Graph Convolutional Encoders for Structured Data to Text Generationcited by 27
  19. 2020Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-trainingcited by 24
  20. 2021Few-Shot Table-to-Text Generation with Prototype Memorycited by 21
  21. 2019A Hierarchical Model for Data-to-Text Generationcited by 20
  22. 2022Data-to-text Generation with Variational Sequential Planningcited by 17
  23. 2021EventNarrative: A large-scale Event-centric Dataset for Knowledge Graph-to-Text Generationcited by 14
  24. 2018Bootstrapping Generators from Noisy Datacited by 11
  25. 2022Self-supervised Graph Masking Pre-training for Graph-to-Text Generationcited by 8
  26. 2023Faithful Low-Resource Data-to-Text Generation through Cycle Trainingcited by 7
  27. 2022Abstractive Summarization Guided by Latent Hierarchical Document Structurecited by 4
  28. 2023Towards Table-to-Text Generation with Pretrained Language Model: A Table Structure Understanding and Text Deliberating Approachcited by 4
  29. 2023Generating Faithful Text From a Knowledge Graph with Noisy Reference Textcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 1.

5 papers.

  1. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  2. 2024FACTOID: FACtual enTailment fOr hallucInation Detectioncited by 10
  3. 2025HalluClean: A Unified Framework to Combat Hallucinations in LLMscited by 1
  4. 2024Multilingual Fine-Grained News Headline Hallucination Detectioncited by 0
  5. 2026Attribution Techniques for Mitigating Hallucinated Information in RAG Systems: A Surveycited by 0