mlsanityData to 14 Oct 2026

Factuality Enhanced Language Models for Open-Ended Text Generation

arXiv 2206.04624 ↗First posted 9 Jun 2022cs.CL cs.AI cs.CYcited by 202

Not on the map. Why?

Abstract (arXiv)

Pretrained language models (LMs) are susceptible to generate text with nonfactual information. In this work, we measure and improve the factual accuracy of large-scale LMs for open-ended text generation. We design the FactualityPrompts test set and metrics to measure the factuality of LM generations. Based on that, we study the factual accuracy of LMs with parameter sizes ranging from 126M to 530B. Interestingly, we find that larger LMs are more factual than smaller ones, although a previous study suggests that larger LMs can be less truthful in terms of misconceptions. In addition, popular sampling algorithms (e.g., top-p) in open-ended text generation can harm the factuality due to the ''uniform randomness'' introduced at every sampling step. We propose the factual-nucleus sampling algorithm that dynamically adapts the randomness to improve the factuality of generation while maintaining quality. Furthermore, we analyze the inefficiencies of the standard training method in learning correct associations between entities from factual text corpus (e.g., Wikipedia). We propose a factuality-enhanced training method that uses TopicPrefix for better awareness of facts and sentence completion as the training objective, which can vastly reduce the factual errors. We release our code and FactualityPrompts benchmark at: https://github.com/nayeon7lee/FactualityPrompt.

Key facts. Cited by 202 papers in the map (data through 14 Oct 2026). Builds on 70 of its 80 references that are in the map. First posted on arXiv 9 Jun 2022.

Builds on

70 of 80 references.

References · 70

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  10. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  11. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  12. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
38 more
  1. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  3. 2019Language Models as Knowledge Bases?cited by 1,936
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  6. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  7. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  8. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  9. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  10. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  11. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  12. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  13. 2019How Can We Know What Language Models Know?cited by 959
  14. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  15. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  16. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  17. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  18. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
  19. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  20. 2019Defending Against Neural Fake Newscited by 559
  21. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  22. 2021Editing Factual Knowledge in Language Modelscited by 553
  23. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  24. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
  25. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 293
  26. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  27. 2017Challenges in Data-to-Document Generationcited by 226
  28. 2021Towards Continual Knowledge Learning of Language Modelscited by 174
  29. 2017Faithful to the Original: Fact Aware Neural Abstractive Summarizationcited by 141
  30. 2020A Survey of Knowledge-Enhanced Text Generationcited by 120
  31. 2021CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarizationcited by 120
  32. 2021Entity-level Factual Consistency of Abstractive Text Summarizationcited by 118
  33. 2021Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Groundingcited by 106
  34. 2019Assessing The Factual Accuracy of Generated Textcited by 103
  35. 2019Evaluating Commonsense in Pre-trained Language Modelscited by 95
  36. 2021The Factual Inconsistency Problem in Abstractive Text Summarization: A Surveycited by 79
  37. 2021Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selectioncited by 76
  38. 2019Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generationcited by 75

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 56; 2024: 80; 2025: 46; 2026: 17.

202 papers.

Sort:

By area:

  1. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  4. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  5. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  6. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  7. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 478
  8. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
  9. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  10. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  11. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  12. 2023Large Language Model Alignment: A Surveycited by 243
38 more
  1. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 224
  2. 2025Why Language Models Hallucinatecited by 220
  3. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 206
  4. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  5. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  6. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  7. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 150
  8. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 131
  9. 2024Language Models with Conformal Factuality Guaranteescited by 131
  10. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 127
  11. 2023Factuality Challenges in the Era of Large Language Modelscited by 126
  12. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 124
  13. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 111
  14. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 109
  15. 2023Generating Benchmarks for Factuality Evaluation of Language Modelscited by 105
  16. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 99
  17. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 98
  18. 2024Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Modelscited by 94
  19. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  20. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 88
  21. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 87
  22. 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 86
  23. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 84
  24. 2023Alignment for Honestycited by 74
  25. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  26. 2023Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Studycited by 63
  27. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignmentcited by 56
  28. 2023Why Does ChatGPT Fall Short in Providing Truthful Answers?cited by 54
  29. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 51
  30. 2023Resolving Knowledge Conflicts in Large Language Modelscited by 50
  31. 2024Learning to Trust Your Feelings: Leveraging Self-awareness in LLMs for Hallucination Mitigationcited by 44
  32. 2024Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Modelscited by 44
  33. 2023Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generatorscited by 43
  34. 2025Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decodingcited by 43
  35. 2023"According to ...": Prompting Language Models Improves Quoting from Pre-Training Datacited by 42
  36. 2023A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detectioncited by 42
  37. 2022Measuring Reliability of Large Language Models through Semantic Consistencycited by 38
  38. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 37

50 most cited of 202 All 202 (JSON)