# FactReasoner: A Probabilistic Approach to Long-Form Factuality Assessment for Large Language Models

arXiv: https://arxiv.org/abs/2502.18573 · HTML: https://mlsanity.com/paper/2502.18573/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 25 Feb 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 15 of 15)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models](https://mlsanity.com/paper/2401.01313/) (2024)
- [Language Model Cascades](https://mlsanity.com/paper/2207.10342/) (2022)
- [A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Models](https://mlsanity.com/paper/2405.09589/) (2024)
- [ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM](https://mlsanity.com/paper/2408.12076/) (2024)
- [FELM: Benchmarking Factuality Evaluation of Large Language Models](https://mlsanity.com/paper/2310.00741/) (2023)
- [DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2407.17023/) (2024)
- [Evaluating Hallucinations in Chinese Large Language Models](https://mlsanity.com/paper/2310.03368/) (2023)
- [FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation](https://mlsanity.com/paper/2410.22257/) (2024)
- [Who's Who: Large Language Models Meet Knowledge Conflicts in Practice](https://mlsanity.com/paper/2410.15737/) (2024)

## Built on by (most cited first, 2 of 2)

- [PRISMM-Bench: A Benchmark of Peer-Review Grounded Multimodal Inconsistencies](https://mlsanity.com/paper/2510.16505/) (2025)
- [Interpreting LLM-as-a-Judge Policies via Verifiable Global Explanations](https://mlsanity.com/paper/2510.08120/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have achieved remarkable success in generative tasks, yet they often fall short in ensuring the factual accuracy of their outputs, thus limiting their reliability in real-world applications where correctness is critical. In this paper, we present FactReasoner, a novel neuro-symbolic based factuality assessment framework that employs probabilistic reasoning to evaluate the truthfulness of long-form generated responses. FactReasoner decomposes a response into atomic units, retrieves relevant contextual information from external knowledge sources, and models the logical relationships (e.g., entailment, contradiction) between these units and their contexts using probabilistic encodings. It then estimates the posterior probability that each atomic unit is supported by the retrieved evidence. Our experiments on both labeled and unlabeled benchmark datasets demonstrate that FactReasoner often outperforms state-of-the-art prompt-based methods in terms of factual precision and recall. Our open-source implementation is publicly available at: https://github.com/IBM/FactReasoner.