# Graph Reasoning for Question Answering with Triplet Retrieval

arXiv: https://arxiv.org/abs/2305.18742 · HTML: https://mlsanity.com/paper/2305.18742/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 40 of its 53 references that are in the map.
- First posted on arXiv 30 May 2023.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 30 of 40)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering](https://mlsanity.com/paper/2104.06378/) (2021)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning](https://mlsanity.com/paper/1909.02151/) (2019)
- [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://mlsanity.com/paper/2210.03849/) (2022)
- [GreaseLM: Graph REASoning Enhanced Language Models for Question Answering](https://mlsanity.com/paper/2201.08860/) (2022)
- [Deep Bidirectional Language-Knowledge Graph Pretraining](https://mlsanity.com/paper/2210.09338/) (2022)
- [Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering](https://mlsanity.com/paper/2005.00646/) (2020)
- [JAKET: Joint Pre-training of Knowledge Graph and Language Understanding](https://mlsanity.com/paper/2010.00796/) (2020)
- [UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering](https://mlsanity.com/paper/2012.14610/) (2020)
- [Improving Natural Language Inference Using External Knowledge in the Science Questions Domain](https://mlsanity.com/paper/1809.05724/) (2018)
- [Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering](https://mlsanity.com/paper/2005.00691/) (2020)

## Built on by (most cited first, 30 of 34)

- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning](https://mlsanity.com/paper/2310.01061/) (2023)
- [Graph Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2408.08921/) (2024)
- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.20724/) (2024)
- [Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering](https://mlsanity.com/paper/2404.14741/) (2024)
- [Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models](https://mlsanity.com/paper/2410.13080/) (2024)
- [GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation](https://mlsanity.com/paper/2502.01113/) (2025)
- [FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answering](https://mlsanity.com/paper/2405.13873/) (2024)
- [Dual Reasoning: A GNN-LLM Collaborative Framework for Knowledge Graph Question Answering](https://mlsanity.com/paper/2406.01145/) (2024)
- [Question-guided Knowledge Graph Re-scoring and Injection for Knowledge Graph Question Answering](https://mlsanity.com/paper/2410.01401/) (2024)
- [RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answering](https://mlsanity.com/paper/2410.22353/) (2024)
- [Large Language Models Can Better Understand Knowledge Graphs Than We Thought](https://mlsanity.com/paper/2402.11541/) (2024)
- [KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model's Reasoning Path Aggregation](https://mlsanity.com/paper/2412.20995/) (2024)
- [Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications](https://mlsanity.com/paper/2510.21131/) (2025)
- [Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering](https://mlsanity.com/paper/2506.09645/) (2025)
- [Combining LLMs and Knowledge Graphs to Reduce Hallucinations in Question Answering](https://mlsanity.com/paper/2409.04181/) (2024)
- [RJE: A Retrieval-Judgment-Exploration Framework for Efficient Knowledge Graph Question Answering with LLMs](https://mlsanity.com/paper/2510.01257/) (2025)
- [How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation](https://mlsanity.com/paper/2510.21148/) (2025)
- [ConceptFormer: Towards Efficient Use of Knowledge-Graph Embeddings in Large Language Models](https://mlsanity.com/paper/2504.07624/) (2025)
- [GPR: Empowering Generation with Graph-Pretrained Retriever](https://mlsanity.com/paper/2506.00261/) (2025)
- [GraphextQA: A Benchmark for Evaluating Graph-Enhanced Large Language Models](https://mlsanity.com/paper/2310.08487/) (2023)
- [Pyramid-Driven Alignment: Pyramid Principle Guided Integration of Large Language Models and Knowledge Graphs](https://mlsanity.com/paper/2410.12298/) (2024)
- [ADKGD: Anomaly Detection in Knowledge Graphs with Dual-Channel Training](https://mlsanity.com/paper/2501.07078/) (2025)
- [Walk&Retrieve: Simple Yet Effective Zero-shot Retrieval-Augmented Generation via Knowledge Graph Walks](https://mlsanity.com/paper/2505.16849/) (2025)
- [DistRAG: Towards Distance-Based Spatial Reasoning in LLMs](https://mlsanity.com/paper/2506.03424/) (2025)
- [MKG-RAG-Bench: Benchmarking Retrieval in Multimodal Knowledge Graph-Augmented Generation](https://mlsanity.com/paper/2606.26458/) (2026)
- [Guided Navigation in Knowledge-Dense Environments: Structured Semantic Exploration with Guidance Graphs](https://mlsanity.com/paper/2508.10012/) (2025)
- [GRIL: Knowledge Graph Retrieval-Integrated Learning with Large Language Models](https://mlsanity.com/paper/2509.16502/) (2025)
- [Enhancing LLM Generation with Knowledge Hypergraph for Evidence-Based Medicine](https://mlsanity.com/paper/2503.16530/) (2025)

## Abstract (arXiv)

Answering complex questions often requires reasoning over knowledge graphs (KGs). State-of-the-art methods often utilize entities in questions to retrieve local subgraphs, which are then fed into KG encoder, e.g. graph neural networks (GNNs), to model their local structures and integrated into language models for question answering. However, this paradigm constrains retrieved knowledge in local subgraphs and discards more diverse triplets buried in KGs that are disconnected but useful for question answering. In this paper, we propose a simple yet effective method to first retrieve the most relevant triplets from KGs and then rerank them, which are then concatenated with questions to be fed into language models. Extensive results on both CommonsenseQA and OpenbookQA datasets show that our method can outperform state-of-the-art up to 4.6% absolute accuracy.