mlsanityData to 14 Oct 2026

FactAlign: Long-form Factuality Alignment of Large Language Models

arXiv 2410.01691 ↗First posted 2 Oct 2024cs.CL cs.AIcited by 12

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Large language models have demonstrated significant potential as the next-generation information access engines. However, their reliability is hindered by issues of hallucination and generating non-factual content. This is particularly problematic in long-form responses, where assessing and ensuring factual accuracy is complex. In this paper, we address this gap by proposing FactAlign, a novel alignment framework designed to enhance the factuality of LLMs' long-form responses while maintaining their helpfulness. We introduce fKTO, a fine-grained, sentence-level alignment algorithm that extends the Kahneman-Tversky Optimization (KTO) alignment method. Leveraging recent advances in automatic factuality evaluation, FactAlign utilizes fine-grained factuality assessments to guide the alignment process. Our experiments on open-domain prompts and information-seeking questions demonstrate that FactAlign significantly improves the factual accuracy of LLM responses while also improving their helpfulness. Further analyses identify that FactAlign is capable of training LLMs to provide more information without losing factual precision, thus improving the factual F1 score. Our source code, datasets, and trained models are publicly available at https://github.com/MiuLab/FactAlign

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 27 of its 34 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

27 of 34 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinate8 Mar
  2. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning25 Dec
  3. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  9. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  10. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  11. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  12. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
12 more
  1. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  2. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  3. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  4. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  5. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  6. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 401
  7. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 232
  8. 2023Fine-tuning Language Models for Factualitycited by 221
  9. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
  10. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 149
  11. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 131
  12. 2024Merging Facts, Crafting Fallacies: Evaluating the Contradictory Nature of Aggregated Factual Claims in Long-Form Generationscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 4.

12 papers.

  1. 2025The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Inputcited by 52
  2. 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agentscited by 50
  3. 2025MedScore: Generalizable Factuality Evaluation of Free-Form Medical Answers by Domain-adapted Claim Decomposition and Verificationcited by 5
  4. 2025Knowledge-Level Consistency Reinforcement Learning: Dual-Fact Alignment for Long-Form Factualitycited by 4
  5. 2025MR-Align: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Modelscited by 4
  6. 2025Atomic Consistency Preference Optimization for Long-Form Question Answeringcited by 1
  7. 2025An Explainable Emotion Alignment Framework for LLM-Empowered Agent in Metaverse Service Ecosystemcited by 1
  8. 2025Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generationcited by 1
  9. 2025Mathematical Analysis of Hallucination Dynamics in Large Language Models: Uncertainty Quantification, Advanced Decoding, and Principled Mitigationcited by 0
  10. 2026PRISM: Probability Reallocation with In-Span Masking for Knowledge-Sensitive Alignmentcited by 0
  11. 2026An Information-Geometric Framework for Stability Analysis of Large Language Models under Entropic Stresscited by 0
  12. 2026Building Reliable Long-Form Generation via Hallucination Rejection Samplingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Curious Case of Factual (Mis)Alignment between LLMs' Short- and Long-Form Answers
  2. 2024UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models
  3. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
  4. 2024Long-form factuality in large language models
  5. 2023Aligning Large Language Models with Human: A Survey
  6. 2024Beyond Under-Alignment: Atomic Preference Enhanced Factuality Tuning for Large Language Models
  7. 2026Do Factuality Methods Make LLMs More Factual? Reproducibility and Fair Comparison
  8. 2023Trusted Source Alignment in Large Language Models