mlsanityData to 7 Oct 2026

Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphs

arXiv 2402.11199 ↗First posted 17 Feb 2024cs.CLcited by 32

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Large language models (LLMs) demonstrate strong reasoning abilities when prompted to generate chain-of-thought (CoT) explanations alongside answers. However, previous research on evaluating LLMs has solely focused on answer accuracy, neglecting the correctness of the generated CoT. In this paper, we delve deeper into the CoT reasoning capabilities of LLMs in multi-hop question answering by utilizing knowledge graphs (KGs). We propose a novel discriminative and generative CoT evaluation paradigm to assess LLMs' knowledge of reasoning and the accuracy of the generated CoT. Through experiments conducted on 5 different families of LLMs across 2 multi-hop question-answering datasets, we find that LLMs possess sufficient knowledge to perform reasoning. However, there exists a significant disparity between answer accuracy and faithfulness of the CoT reasoning generated by LLMs, indicating that they often arrive at correct answers through incorrect reasoning.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 41 references that are in the map. First posted on arXiv 17 Feb 2024.

Builds on

17 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning2 Oct

Other references · 16

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2023Qwen Technical Reportcited by 3,273
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  6. 2023Mistral 7Bcited by 3,065
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  8. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  9. 2023Faithful Chain-of-Thought Reasoningcited by 285
  10. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  11. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
  12. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
4 more
  1. 2022Faithful Reasoning Using Large Language Modelscited by 130
  2. 2023LLMs cannot find reasoning errors, but can correct them given the error locationcited by 85
  3. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Promptingcited by 77
  4. 2023FELM: Benchmarking Factuality Evaluation of Large Language Modelscited by 66

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 15; 2026: 12.

32 papers.

Sort:

By area:

  1. 2025Confidence Improves Self-Consistency in LLMscited by 111
  2. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 85
  3. 2025Evaluating Step-by-step Reasoning Traces: A Surveycited by 62
  4. 2024An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Modelscited by 49
  5. 2024Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Modelscited by 47
  6. 2025Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Trainingcited by 29
  7. 2025From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?cited by 21
  8. 2024TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generationcited by 16
  9. 2024OCEAN: Offline Chain-of-thought Evaluation and Alignment in Large Language Modelscited by 12
  10. 2025DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Actioncited by 11
  11. 2025Do LLM Agents Know How to Ground, Recover, and Assess? A Benchmark for Epistemic Competence in Information-Seeking Agentscited by 7
  12. 2025TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM Reasoningcited by 6
20 more
  1. 2026Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertaintycited by 6
  2. 2025Are Language Models Efficient Reasoners? A Perspective from Logic Programmingcited by 4
  3. 2026Evaluating Chain-of-Thought Reasoning through Reusability and Verifiabilitycited by 4
  4. 2026Compress the Easy, Explore the Hard: Difficulty-Aware Entropy Regularization for Efficient LLM Reasoningcited by 4
  5. 2025Are Large Language Models Good Temporal Graph Learners?cited by 3
  6. 2026Pix2Fact: When Vision Is Not Enough -- Benchmarking Fine-Grained VQA with Web Verification on High-Resolution Real-World Scenescited by 3
  7. 2026TDA-RC: Task-Driven Alignment for Knowledge-Based Reasoning Chains in Large Language Modelscited by 2
  8. 2025KG-CQR: Leveraging Structured Relation Representations in Knowledge Graphs for Contextual Query Retrievalcited by 1
  9. 2025RCScore: Quantifying Response Consistency in Large Language Modelscited by 1
  10. 2025DecompSR: A dataset for decomposed analyses of compositional multihop spatial reasoningcited by 1
  11. 2025Reasoning Relay: Evaluating Stability and Interchangeability of Large Language Models in Mathematical Reasoningcited by 1
  12. 2026Beyond Factual Accuracy: Evaluating Global Reasoning Integrity in RAG Systems with LogicScorecited by 1
  13. 2026The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearningcited by 1
  14. 2026GraphSteal: Structural Knowledge Stealing from Graph RAG via Traversal Reconstructioncited by 1
  15. 2026EVL-MCoT: Enhanced Vision-Language Multi-CoT for Harmful Meme Detectioncited by 1
  16. 2026Right Answer, Wrong Method: Shortcut Hacking Misleads the Evaluation of LLM Reasoning on Frontier Science Benchmarkscited by 1
  17. 2025SHARP: Shared State Reduction for Efficient Matching of Sequential Patternscited by 0
  18. 2025ProofSketch: Efficient Verified Reasoning for Large Language Modelscited by 0
  19. 2026Better Think Thrice: Learning to Reason Causally with Double Counterfactual Consistencycited by 0
  20. 2026The Answer Path and the Grounding Instruction in LLM Question Answering over Knowledge Graphscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration
  2. 2023Graph Elicitation for Guiding Multi-Step Reasoning in Large Language Models
  3. 2023Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning
  4. 2023Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answering
  5. 2025Reasoning with Graphs: Structuring Implicit Knowledge to Enhance LLMs Reasoning
  6. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thought
  7. 2026Omanic: Towards Step-wise Evaluation of Multi-hop Reasoning in Large Language Models
  8. 2024A Hopfieldian View-based Interpretation for Chain-of-Thought Reasoning