Beyond Factual Accuracy: Evaluating Global Reasoning Integrity in RAG Systems with LogicScore
On the map: LLM reasoning → Knowledge- and tool-augmented reasoning
also in 1 more
Abstract (arXiv)
Current evaluation methods for Retrieval Augmented Generation (RAG) suffer from textitfactual myopia: they relentlessly emphasize factual accuracy yet neglect global logical integrity in long-form answer generation. This drives models to force unnatural connections, producing factually grounded yet logically incoherent responses with unaddressed gaps, ambiguous links, or redundant premises. To mitigate this, we present textscLogicScore, shifting from local, fact-by-fact assessment to rigorous global reasoning scrutiny. Grounded in Horn Rules, our approach integrates a backward verification mechanism to systematically evaluate three key reasoning dimensions: textitCompleteness (logically sound deduction), textitEssentiality (non-redundancy), and textitDeterminateness (consistent answer entailment). Extensive experiments across three multi-hop QA datasets (HotpotQA, MusiQue, and 2WikiMultiHopQA) and over 20 LLMs (including GPT-5, Gemini-3-Pro, LLaMA3, and task-specific tuned models) reveal a critical capability gap: leading models often achieve high factual accuracy (e.g., 92.85% precision for Gemini-3 Pro) but struggle with global reasoning quality (e.g., 35.11% Essentiality for Gemini-3 Pro). Our work establishes a robust standard for logical evaluation, highlighting the need to prioritize reasoning coherence alongside factual grounding in LLM development.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 45 references that are in the map. First posted on arXiv 21 Jan 2026.
Builds on
34 of 45 references.
Closest ancestors · 2
Same direction, newest first
- 2024Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning23 May
- 2024Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphs17 Feb
Other references · 32
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
- 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
- 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
- 2021Measuring Attribution in Natural Language Generation Modelscited by 253
20 more
- 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 175
- 2023Complex Claim Verification with Evidence Retrieved in the Wildcited by 110
- 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
- 2024LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QAcited by 78
- 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
- 2024A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chainscited by 63
- 2024Attribute First, then Generate: Locally-attributable Grounded Text Generationcited by 54
- 2023SocREval: Large Language Models with the Socratic Method for Reference-Free Reasoning Evaluationcited by 27
- 2024Improving Attributed Text Generation of Large Language Models via Preference Learningcited by 23
- 2024Atomic Fact Decomposition Helps Attributed Question Answeringcited by 21
- 2025SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Modelscited by 19
- 2025CiteEval: Principle-Driven Citation Evaluation for Source Attributioncited by 19
- 2024Can LLMs Evaluate Complex Attribution in QA? Automatic Benchmarking using Knowledge Graphscited by 16
- 2024Learning to Generate Answers with Citations via Factual Consistency Modelscited by 13
- 2025LAQuer: Localized Attribution Queries in Content-grounded Generationcited by 13
- 2024Learning Fine-Grained Grounded Citations for Attributed Large Language Modelscited by 12
- 2025Temporalizing Confidence: Evaluation of Chain-of-Thought Reasoning with Signal Temporal Logiccited by 11
- 2024A Survey on Symbolic Knowledge Distillation of Large Language Modelscited by 9
- 2024MiCEval: Unveiling Multimodal Chain of Thought's Quality via Image Description and Reasoning Stepscited by 8
- 2025Document Attribution: Examining Citation Relationships using Large Language Modelscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation
- 2025From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs
- 2026Reason and Verify: A Framework for Faithful Retrieval-Augmented Generation
- 2025Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval Augmented Generation
- 2026Why RAGs Hallucinate: Penalty-Aware Evaluation of Retrieval-Augmented Generation Systems with Knowledge-Gap Canaries
- 2025PrismRAG: Boosting RAG Factuality with Distractor Resilience and Strategized Reasoning
- 2025Frustratingly Simple Retrieval Improves Challenging, Reasoning-Intensive Benchmarks
- 2026ReliableRAG: Combating Misinformation in Retrieval-Augmented Generation via Reliability-Guided Reasoning Chains