# Don't Forget to Connect! Improving RAG with Graph-based Reranking

arXiv: https://arxiv.org/abs/2405.18414 · HTML: https://mlsanity.com/paper/2405.18414/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 28 May 2024.

## Placed in

- Retrieval-augmented generation / [Knowledge-graph and structured-knowledge RAG](https://mlsanity.com/rag/knowledge-graph-and-structured-knowledge-rag/)

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering](https://mlsanity.com/paper/1911.10470/) (2019)
- [Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering](https://mlsanity.com/paper/2210.02627/) (2022)
- [Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!](https://mlsanity.com/paper/2303.08559/) (2023)
- [RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses](https://mlsanity.com/paper/2210.10634/) (2022)
- [Re2G: Retrieve, Rerank, Generate](https://mlsanity.com/paper/2207.06300/) (2022)
- [Zero-Shot Listwise Document Reranking with a Large Language Model](https://mlsanity.com/paper/2305.02156/) (2023)
- [Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Family](https://mlsanity.com/paper/2303.07992/) (2023)
- [KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering](https://mlsanity.com/paper/2110.04330/) (2021)
- [Graph Pre-training for AMR Parsing and Generation](https://mlsanity.com/paper/2203.07836/) (2022)
- [Improving Pairwise Ranking for Multi-label Image Classification](https://mlsanity.com/paper/1704.03135/) (2017)
- [Semantic Representation for Dialogue Modeling](https://mlsanity.com/paper/2105.10188/) (2021)
- [Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answering](https://mlsanity.com/paper/2210.02933/) (2022)

## Built on by (most cited first, 30 of 49)

- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [RAG vs. GraphRAG: A Systematic Evaluation and Key Insights](https://mlsanity.com/paper/2502.11371/) (2025)
- [Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.08826/) (2025)
- [GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation](https://mlsanity.com/paper/2502.01113/) (2025)
- [On the Structural Memory of LLM Agents](https://mlsanity.com/paper/2412.15266/) (2024)
- [HybGRAG: Hybrid Retrieval-Augmented Generation on Textual and Relational Knowledge Bases](https://mlsanity.com/paper/2412.16311/) (2024)
- [Empowering GraphRAG with Knowledge Filtering and Integration](https://mlsanity.com/paper/2503.13804/) (2025)
- [HyperMem: Hypergraph Memory for Long-Term Conversations](https://mlsanity.com/paper/2604.08256/) (2026)
- [Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs](https://mlsanity.com/paper/2410.11001/) (2024)
- [Causal Graphs Meet Thoughts: Enhancing Complex Reasoning in Graph-Augmented LLMs](https://mlsanity.com/paper/2501.14892/) (2025)
- [CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning](https://mlsanity.com/paper/2511.18659/) (2025)
- [OmniQuery: Contextually Augmenting Captured Multimodal Memory to Enable Personal Question Answering](https://mlsanity.com/paper/2409.08250/) (2024)
- [ShizishanGPT: An Agricultural Large Language Model Integrating Tools and Resources](https://mlsanity.com/paper/2409.13537/) (2024)
- [Align-GRAG: Anchor and Rationale Guided Dual Alignment for Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.16237/) (2025)
- [EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notes](https://mlsanity.com/paper/2509.00877/) (2025)
- [SMART-RAG: Selection using Determinantal Matrices for Augmented Retrieval](https://mlsanity.com/paper/2409.13992/) (2024)
- [LEGO-GraphRAG: Modularizing Graph-based Retrieval-Augmented Generation for Design Space Exploration](https://mlsanity.com/paper/2411.05844/) (2024)
- [POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Models](https://mlsanity.com/paper/2505.06579/) (2025)
- [Hybrid-NL2SVA: Integrating RAG and Finetuning for LLM-based NL2SVA](https://mlsanity.com/paper/2506.21569/) (2025)
- [A Deep Dive into Retrieval-Augmented Generation for Code Completion: Experience on WeChat](https://mlsanity.com/paper/2507.18515/) (2025)
- [SciRerankBench: Benchmarking Rerankers Towards Scientific Retrieval-Augmented Generated LLMs](https://mlsanity.com/paper/2508.08742/) (2025)
- [Document Intelligence in the Era of Large Language Models: A Survey](https://mlsanity.com/paper/2510.13366/) (2025)
- [AcademicEval: Live Long-Context LLM Benchmark](https://mlsanity.com/paper/2510.17725/) (2025)
- [What External Knowledge is Preferred by LLMs? Characterizing and Exploring Chain of Evidence in Imperfect Context for Multi-Hop QA](https://mlsanity.com/paper/2412.12632/) (2024)
- [A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challenges](https://mlsanity.com/paper/2507.21117/) (2025)
- [The Evolution of Reranking Models in Information Retrieval: From Heuristic Methods to Large Language Models](https://mlsanity.com/paper/2512.16236/) (2025)
- [NeocorRAG: Less Irrelevant Information, More Explicit Evidence, and More Effective Recall via Evidence Chains](https://mlsanity.com/paper/2604.27852/) (2026)
- [Gumbel Reranking: Differentiable End-to-End Reranker Optimization](https://mlsanity.com/paper/2502.11116/) (2025)
- [Finetune-RAG: Fine-Tuning Language Models to Resist Hallucination in Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.10792/) (2025)
- [Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers](https://mlsanity.com/paper/2512.10422/) (2025)

## Abstract (arXiv)

Retrieval Augmented Generation (RAG) has greatly improved the performance of Large Language Model (LLM) responses by grounding generation with context from existing documents. These systems work well when documents are clearly relevant to a question context. But what about when a document has partial information, or less obvious connections to the context? And how should we reason about connections between documents? In this work, we seek to answer these two core questions about RAG generation. We introduce G-RAG, a reranker based on graph neural networks (GNNs) between the retriever and reader in RAG. Our method combines both connections between documents and semantic information (via Abstract Meaning Representation graphs) to provide a context-informed ranker for RAG. G-RAG outperforms state-of-the-art approaches while having smaller computational footprint. Additionally, we assess the performance of PaLM 2 as a reranker and find it to significantly underperform G-RAG. This result emphasizes the importance of reranking for RAG even when using Large Language Models.