# LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond

arXiv: https://arxiv.org/abs/2305.14540 · HTML: https://mlsanity.com/paper/2305.14540/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 17 of its 50 references that are in the map.
- First posted on arXiv 23 May 2023.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 17 of 17)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Can language models learn from explanations in context?](https://mlsanity.com/paper/2204.02329/) (2022)
- [The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning](https://mlsanity.com/paper/2205.03401/) (2022)
- [Evaluating Factuality in Generation with Dependency-level Entailment](https://mlsanity.com/paper/2010.05478/) (2020)
- [Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors](https://mlsanity.com/paper/2205.12854/) (2022)
- [ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts](https://mlsanity.com/paper/2210.12467/) (2022)
- [Multi-LexSum: Real-World Summaries of Civil Rights Lawsuits at Multiple Granularities](https://mlsanity.com/paper/2206.10883/) (2022)
- [PREME: Preference-based Meeting Exploration through an Interactive Questionnaire](https://mlsanity.com/paper/2205.02370/) (2022)
- [The Spotify Podcast Dataset](https://mlsanity.com/paper/2004.04270/) (2020)

## Built on by (most cited first, 30 of 38)

- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models](https://mlsanity.com/paper/2501.09686/) (2025)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models](https://mlsanity.com/paper/2402.04614/) (2024)
- [Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment](https://mlsanity.com/paper/2311.08596/) (2023)
- [Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization](https://mlsanity.com/paper/2311.09184/) (2023)
- [Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems](https://mlsanity.com/paper/2401.05778/) (2024)
- [Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices](https://mlsanity.com/paper/2403.12503/) (2024)
- [Beyond the Chat: Executable and Verifiable Text-Editing with LLMs](https://mlsanity.com/paper/2309.15337/) (2023)
- [Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AI](https://mlsanity.com/paper/2309.12444/) (2023)
- [How Far Are We from Intelligent Visual Deductive Reasoning?](https://mlsanity.com/paper/2403.04732/) (2024)
- [Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machina](https://mlsanity.com/paper/2410.19599/) (2024)
- [LeanReasoner: Boosting Complex Logical Reasoning with Lean](https://mlsanity.com/paper/2403.13312/) (2024)
- [Probing the Multi-turn Planning Capabilities of LLMs via 20 Question Games](https://mlsanity.com/paper/2310.01468/) (2023)
- [Are Large Language Models Reliable Judges? A Study on the Factuality Evaluation Capabilities of LLMs](https://mlsanity.com/paper/2311.00681/) (2023)
- [RvLLM: LLM Runtime Verification with Domain Knowledge](https://mlsanity.com/paper/2505.18585/) (2025)
- [ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?](https://mlsanity.com/paper/2311.16989/) (2023)
- [Evaluating LLM Reasoning in the Operations Research Domain with ORQA](https://mlsanity.com/paper/2412.17874/) (2024)
- [AMRFact: Enhancing Summarization Factuality Evaluation with AMR-Driven Negative Samples Generation](https://mlsanity.com/paper/2311.09521/) (2023)
- [SQLucid: Grounding Natural Language Database Queries with Interactive Explanations](https://mlsanity.com/paper/2409.06178/) (2024)
- [DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence](https://mlsanity.com/paper/2509.04499/) (2025)
- [ERBench: An Entity-Relationship based Automatically Verifiable Hallucination Benchmark for Large Language Models](https://mlsanity.com/paper/2403.05266/) (2024)
- [Automatic Generation of Model and Data Cards: A Step Towards Responsible AI](https://mlsanity.com/paper/2405.06258/) (2024)
- [Rationale-Aware Answer Verification by Pairwise Self-Evaluation](https://mlsanity.com/paper/2410.04838/) (2024)
- [Integration of cognitive tasks into artificial general intelligence test for large models](https://mlsanity.com/paper/2402.02547/) (2024)
- [Integrating Large Language Models and Knowledge Graphs for Extraction and Validation of Textual Test Data](https://mlsanity.com/paper/2408.01700/) (2024)
- [Automated Researchers Can Mitigate Well-characterized Alignment Failures](https://mlsanity.com/paper/2608.28945/) (2026)
- [FaNS: a Facet-based Narrative Similarity Metric](https://mlsanity.com/paper/2309.04823/) (2023)
- [Less is More for Improving Automatic Evaluation of Factual Consistency](https://mlsanity.com/paper/2404.06579/) (2024)
- [HIDE and Seek: Detecting Hallucinations in Language Models via Decoupled Representations](https://mlsanity.com/paper/2506.17748/) (2025)

## Abstract (arXiv)

With the recent appearance of LLMs in practical settings, having methods that can effectively detect factual inconsistencies is crucial to reduce the propagation of misinformation and improve trust in model outputs. When testing on existing factual consistency benchmarks, we find that a few large language models (LLMs) perform competitively on classification benchmarks for factual inconsistency detection compared to traditional non-LLM methods. However, a closer analysis reveals that most LLMs fail on more complex formulations of the task and exposes issues with existing evaluation benchmarks, affecting evaluation precision. To address this, we propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits. This new benchmark is 20 times more cost-effective per sample than previous benchmarks and highly reproducible, as we estimate inter-annotator agreement at about 0.9. Most LLMs struggle on SummEdits, with performance close to random chance. The best-performing model, GPT-4, is still 8% below estimated human performance, highlighting the gaps in LLMs' ability to reason about facts and detect inconsistencies when they occur.