mlsanityData to 7 Oct 2026

Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation

arXiv 2402.09267 ↗First posted 14 Feb 2024cs.CL cs.AIcited by 79

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Despite showing increasingly human-like abilities, large language models (LLMs) often struggle with factual inaccuracies, i.e. "hallucinations", even when they hold relevant knowledge. To address these hallucinations, current approaches typically necessitate high-quality human factuality annotations. In this work, we explore Self-Alignment for Factuality, where we leverage the self-evaluation capability of an LLM to provide training signals that steer the model towards factuality. Specifically, we incorporate Self-Eval, a self-evaluation component, to prompt an LLM to validate the factuality of its own generated responses solely based on its internal knowledge. Additionally, we design Self-Knowledge Tuning (SK-Tuning) to augment the LLM's self-evaluation ability by improving the model's confidence estimation and calibration. We then utilize these self-annotated responses to fine-tune the model via Direct Preference Optimization algorithm. We show that the proposed self-alignment approach substantially enhances factual accuracy over Llama family models across three key knowledge-intensive tasks on TruthfulQA and BioGEN.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 43 of its 69 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

43 of 69 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback24 May
  2. 2023LIMA: Less Is More for Alignment18 May
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec
  4. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 39

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  10. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  11. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  12. 2022Language Models (Mostly) Know What They Knowcited by 1,783
27 more
  1. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  2. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  4. 2022Galactica: A Large Language Model for Sciencecited by 736
  5. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  6. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  7. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  8. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  9. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  10. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  11. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  12. 2022Self-critiquing models for assisting human evaluatorscited by 372
  13. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  14. 2023Fine-tuning Language Models for Factualitycited by 219
  15. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
  16. 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 141
  17. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  18. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 121
  19. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  20. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 78
  21. 2023Adaptation with Self-Evaluation to Improve Selective Prediction in LLMscited by 49
  22. 2023Rethinking Confidence Calibration for Failure Predictioncited by 43
  23. 2023SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Promptingcited by 39
  24. 2023Llamas Know What GPTs Don't Show: Surrogate Models for Confidence Estimationcited by 38
  25. 2023Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?cited by 36
  26. 2017ConceptNet at SemEval-2017 Task 2: Extending Word Embeddings with Multilingual Relational Knowledgecited by 15
  27. 2022Toward Self-learning End-to-End Task-Oriented Dialog Systemscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 23; 2025: 44; 2026: 12.

79 papers.

Sort:

By area:

  1. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  2. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 119
  3. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  4. 2024A Survey on Self-Evolution of Large Language Modelscited by 86
  5. 2025Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedbackcited by 86
  6. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
  7. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  8. 2024Human-LLM Compound System for Scientific Ideation through Facet Recombination and Novelty Evaluationcited by 39
  9. 2024A Survey of Useful LLM Evaluationcited by 28
  10. 2025Mask-DPO: Generalizable Fine-grained Factuality Alignment of LLMscited by 22
  11. 2024A Survey on the Honesty of Large Language Modelscited by 21
  12. 2025TruthRL: Incentivizing Truthful LLMs via Reinforcement Learningcited by 20
38 more
  1. 2024LoGU: Long-form Generation with Uncertainty Expressionscited by 19
  2. 2024Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulnesscited by 17
  3. 2024Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teachingcited by 17
  4. 2025Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Modelscited by 16
  5. 2025Revisiting Epistemic Markers in Confidence Estimation: Can Markers Accurately Reflect Large Language Models' Uncertainty?cited by 16
  6. 2024Improving Model Factuality with Fine-grained Critique-based Evaluatorcited by 15
  7. 2025Refine Knowledge of Large Language Models via Adaptive Contrastive Learningcited by 14
  8. 2025FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMscited by 14
  9. 2024UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Modelscited by 13
  10. 2025Aligning Large Language Models to Follow Instructions and Hallucinate Less via Effective Data Filteringcited by 12
  11. 2025φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitationcited by 12
  12. 2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
  13. 2024SEER: Self-Aligned Evidence Extraction for Retrieval-Augmented Generationcited by 9
  14. 2025LLMs for Customized Marketing Content Generation and Evaluation at Scalecited by 9
  15. 2025Do Large Language Models Know What They Are Capable Of?cited by 9
  16. 2024On-Policy Self-Alignment with Fine-grained Knowledge Feedback for Hallucination Mitigationcited by 8
  17. 2025Will Pre-Training Ever End? A First Step Toward Next-Generation Foundation MLLMs via Self-Improving Systematic Cognitioncited by 7
  18. 2024LLMs can learn self-restraint through iterative self-reflectioncited by 6
  19. 2025Aligning Large Language Models for Faithful Integrity Against Opposing Argumentcited by 6
  20. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 6
  21. 2025Semantic Voting: A Self-Evaluation-Free Approach for Efficient LLM Self-Improvement on Unverifiable Open-ended Taskscited by 6
  22. 2026Self-Improving Pretraining: using post-trained models to pretrain better modelscited by 6
  23. 2024RAC: Efficient LLM Factuality Correction with Retrieval Augmentationcited by 5
  24. 2024Bridging the Visual Gap: Fine-Tuning Multimodal Models with Knowledge-Adapted Captionscited by 5
  25. 2025Battling Misinformation: An Empirical Study on Adversarial Factuality in Open-Source Large Language Modelscited by 5
  26. 2025Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotationcited by 5
  27. 2025A Survey on Proactive Defense Strategies Against Misinformation in Large Language Modelscited by 5
  28. 2025URPO: A Unified Reward & Policy Optimization Framework for Large Language Modelscited by 5
  29. 2025FACTORY: A Challenging Human-Verified Prompt Set for Long-Form Factualitycited by 5
  30. 2024ContraSolver: Self-Alignment of Language Models by Resolving Internal Preference Contradictionscited by 4
  31. 2024Investigating Representation Universality: Case Study on Genealogical Representationscited by 4
  32. 2025SelfCheck-Eval: A Multi-Module Framework for Zero-Resource Hallucination Detection in Large Language Modelscited by 4
  33. 2025Exploring the Generalizability of Factual Hallucination Mitigation via Enhancing Precise Knowledge Utilizationcited by 4
  34. 2025MSCoT: Structured Chain-of-Thought Generation for Multiple Programming Languagescited by 4
  35. 2025Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysiscited by 3
  36. 2025Knowledge-Level Consistency Reinforcement Learning: Dual-Fact Alignment for Long-Form Factualitycited by 3
  37. 2025Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Modelscited by 3
  38. 2024Probing LLM Hallucination from Within: Perturbation-Driven Approach via Internal Knowledgecited by 2

50 most cited of 79 All 79 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024FLAME: Factuality-Aware Alignment for Large Language Models
  2. 2025Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models
  3. 2024Fine-Grained Self-Endorsement Improves Factuality and Reasoning
  4. 2026Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning
  5. 2026Listen to the Layers: Mitigating Hallucinations with Inter-Layer Disagreement
  6. 2025Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in LLMs
  7. 2025The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality
  8. 2025OnionEval: An Unified Evaluation of Fact-conflicting Hallucination for Small-Large Language Models