mlsanityData to 7 Oct 2026

RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answering

arXiv 2410.22353 ↗First posted 15 Oct 2024cs.IR cs.AI cs.CLcited by 13

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-augmented generation (RAG) has shown promising potential in knowledge intensive question answering (QA). However, existing approaches only consider the query itself, neither specifying the retrieval preferences for the retrievers nor informing the generators of how to refer to the retrieved documents for the answers, which poses a significant challenge to the QA performance. To address these issues, we propose Rule-guided Retrieval-Augmented Generation with LMs, which explicitly introduces rules for in-context learning (RuleRAG-ICL) to guide retrievers to recall related documents in the directions of rules and uniformly guide generators to reason attributed by the same rules. Moreover, most existing RAG datasets were constructed without considering rules and Knowledge Graphs (KGs) are recognized as providing high-quality rules. Therefore, we construct five rule-aware RAG benchmarks for QA, RuleQA, based on KGs to stress the significance of retrieval and reasoning with rules. Experiments on RuleQA demonstrate RuleRAG-ICL improves the retrieval quality of +89.2% in Recall@10 and answer accuracy of +103.1% in Exact Match, and RuleRAG-FT yields more enhancement. In addition, experiments on four existing RAG datasets show RuleRAG is also effective by offering rules in RuleQA to them, further proving the generalization of rule guidance in RuleRAG.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 58 of its 67 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

58 of 67 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 57

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  9. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2019Language Models as Knowledge Bases?cited by 1,922
  12. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
38 more
  1. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  2. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  4. 2023LIMA: Less Is More for Alignmentcited by 1,080
  5. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  6. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  7. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  8. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  9. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  10. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  11. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  12. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  13. 2023Active Retrieval Augmented Generationcited by 506
  14. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  15. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  16. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 326
  17. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  18. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  19. 2017DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoningcited by 265
  20. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  21. 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
  22. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  23. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
  24. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  25. 2018Learning Sequence Encoders for Temporal Knowledge Graph Completioncited by 145
  26. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  27. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  28. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 121
  29. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
  30. 2021TLogic: Temporal Logical Rules for Explainable Link Forecasting on Temporal Knowledge Graphscited by 59
  31. 2023Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generationcited by 54
  32. 2023KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Basescited by 52
  33. 2024How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?cited by 44
  34. 2023Graph Reasoning for Question Answering with Triplet Retrievalcited by 34
  35. 2022Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Modelcited by 32
  36. 2023GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Modelscited by 30
  37. 2024Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMscited by 28
  38. 2024Case-Based or Rule-Based: How Do Transformers Do the Math?cited by 26

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 5.

13 papers.

Sort:

By area:

  1. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  2. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 40
  3. 2025Your RAG is Unfair: Exposing Fairness Vulnerabilities in Retrieval-Augmented Generation via Backdoor Attackscited by 8
  4. 2025SymRAG: Efficient Neuro-Symbolic Retrieval Through Adaptive Query Routingcited by 5
  5. 2024RuAG: Learned-rule-augmented Generation for Large Language Modelscited by 4
  6. 2025Retrieval Augmented Question Answering: When Should LLMs Admit Ignorance?cited by 2
  7. 2025RIMRULE: Improving Tool-Using Language Agents via MDL-Guided Rule Learningcited by 2
  8. 2024A Survey of Query Optimization in Large Language Modelscited by 1
  9. 2026Neurosymbolic Retrievers for Retrieval-augmented Generationcited by 1
  10. 2026PhyNiKCE: A Neurosymbolic Agentic Framework for Autonomous Computational Fluid Dynamicscited by 1
  11. 2026Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decodingcited by 1
  12. 2026Beyond Similarity: Task-Aligned Retrieval for Language Modelscited by 1
1 more
  1. 2026EvoRAG: Making Knowledge Graph-based RAG Automatically Evolve through Feedback-driven Backpropagationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation
  2. 2025Context-Guided Dynamic Retrieval for Improving Generation Quality in RAG Models
  3. 2025T2-RAGBench: Text-and-Table Benchmark for Evaluating Retrieval-Augmented Generation
  4. 2025KERAG: Knowledge-Enhanced Retrieval-Augmented Generation for Advanced Question Answering
  5. 2024KBM: Delineating Knowledge Boundary for Adaptive Retrieval in Large Language Models
  6. 2024RE-RAG: Improving Open-Domain QA Performance and Interpretability with Relevance Estimator in Retrieval-Augmented Generation
  7. 2025MMKB-RAG: A Multi-Modal Knowledge-Based Retrieval-Augmented Generation Framework
  8. 2025On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems