mlsanityData to 7 Oct 2026

Fine-tuning Language Models for Factuality

arXiv 2311.08401 ↗First posted 14 Nov 2023cs.CL cs.AI cs.LGcited by 219

Not on the map. Why?

Cited by 49 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The fluency and creativity of large pre-trained language models (LLMs) have led to their widespread use, sometimes even as a replacement for traditional search engines. Yet language models are prone to making convincing but factually inaccurate claims, often referred to as 'hallucinations.' These errors can inadvertently spread misinformation or harmfully perpetuate misconceptions. Further, manual fact-checking of model responses is a time-consuming process, making human factuality labels expensive to acquire. In this work, we fine-tune language models to be more factual, without human labeling and targeting more open-ended generation settings than past work. We leverage two key recent innovations in NLP to do so. First, several recent works have proposed methods for judging the factuality of open-ended text by measuring consistency with an external knowledge base or simply a large model's confidence scores. Second, the direct preference optimization algorithm enables straightforward fine-tuning of language models on objectives other than supervised imitation, using a preference ranking over possible model responses. We show that learning from automatically generated factuality preference rankings, generated either through existing retrieval systems or our novel retrieval-free approach, significantly improves the factuality (percent of generated claims that are correct) of Llama-2 on held-out topics compared with RLHF or decoding strategies targeted at factuality. At 7B scale, compared to Llama-2-chat, we observe 58% and 40% reduction in factual error rate when generating biographies and answering medical questions, respectively.

Key facts. Cited by 219 papers in the map (data through 7 Oct 2026). Builds on 27 of its 36 references that are in the map. First posted on arXiv 14 Nov 2023.

Builds on

27 of 36 references.

References · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2020Learning to summarize from human feedbackcited by 2,562
  8. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  9. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  10. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  11. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  12. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
15 more
  1. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  2. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  3. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  4. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  5. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  6. 2023How Language Model Hallucinations Can Snowballcited by 310
  7. 2022Prompting GPT-3 To Be Reliablecited by 258
  8. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  9. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  10. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  11. 2021Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarizationcited by 128
  12. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 123
  13. 2023Do Language Models Know When They're Hallucinating References?cited by 115
  14. 2019Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reportscited by 96
  15. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 114; 2025: 71; 2026: 27.

219 papers.

Sort:

By area:

  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  2. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  3. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  4. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 256
  5. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  6. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  7. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
  8. 2025Why Language Models Hallucinatecited by 205
  9. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  10. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  11. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  12. 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 129
38 more
  1. 2025Confidence Improves Self-Consistency in LLMscited by 111
  2. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  3. 2024Large Language Models Must Be Taught to Know What They Don't Knowcited by 99
  4. 2024Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreementcited by 99
  5. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 85
  6. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
  7. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 80
  8. 2024Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluationcited by 79
  9. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 78
  10. 2024Calibrated Self-Rewarding Vision Language Modelscited by 77
  11. 2024Factuality of Large Language Models: A Surveycited by 64
  12. 2024We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMscited by 61
  13. 2024Injecting New Knowledge into Large Language Models via Supervised Fine-Tuningcited by 56
  14. 2024Aligning Language Models to Explicitly Handle Ambiguitycited by 55
  15. 2024Instruction-tuned Language Models are Better Knowledge Learnerscited by 54
  16. 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
  17. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  18. 2024Can AI Assistants Know What They Don't Know?cited by 47
  19. 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
  20. 2024How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Promptscited by 47
  21. 2024FLAME: Factuality-Aware Alignment for Large Language Modelscited by 46
  22. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44
  23. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  24. 2025EvolveSearch: An Iterative Self-Evolving Search Agentcited by 42
  25. 2024Combining Fine-Tuning and LLM-based Agents for Intuitive Smart Contract Auditing with Justificationscited by 41
  26. 2024Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflectioncited by 40
  27. 2023Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectificationcited by 38
  28. 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Modelscited by 38
  29. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  30. 2024LongReward: Improving Long-context Large Language Models with AI Feedbackcited by 34
  31. 2024Teaching Large Language Models to Express Knowledge Boundary from Their Own Signalscited by 32
  32. 2024Calibrating Long-form Generations from Large Language Modelscited by 31
  33. 2024Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferencescited by 30
  34. 2024Improving Uncertainty Quantification in Large Language Models via Semantic Embeddingscited by 30
  35. 2025Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinkingcited by 29
  36. 2025Steering off Course: Reliability Challenges in Steering Language Modelscited by 29
  37. 2025Fine-tuning Large Language Models for Improving Factuality in Legal Question Answeringcited by 28
  38. 2025Mitigating Object Hallucinations via Sentence-Level Early Interventioncited by 28

50 most cited of 219 All 219 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
  2. 2024Long-form factuality in large language models
  3. 2026Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies
  4. 2025Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models
  5. 2025From Parameters to Prompts: Understanding and Mitigating the Factuality Gap between Fine-Tuned LLMs
  6. 2023FELM: Benchmarking Factuality Evaluation of Large Language Models
  7. 2024UFO: a Unified and Flexible Framework for Evaluating Factuality of Large Language Models
  8. 2025FactReasoner: A Probabilistic Approach to Long-Form Factuality Assessment for Large Language Models