mlsanityData to 14 Oct 2026

Rule-Based Explanations for Retrieval-Augmented LLM Systems

arXiv 2510.22689 ↗First posted 26 Oct 2025cs.CLcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

If-then rules are widely used to explain machine learning models; e.g., "if employed = no, then loan application = rejected." We present the first proposal to apply rules to explain the emerging class of large language models (LLMs) with retrieval-augmented generation (RAG). Since RAG enables LLM systems to incorporate retrieved information sources at inference time, rules linking the presence or absence of sources can explain output provenance; e.g., "if a Times Higher Education ranking article is retrieved, then the LLM ranks Oxford first." To generate such rules, a brute force approach would probe the LLM with all source combinations and check if the presence or absence of any sources leads to the same output. We propose optimizations to speed up rule generation, inspired by Apriori-like pruning from frequent itemset mining but redefined within the scope of our novel problem. We conclude with qualitative and quantitative experiments demonstrating our solutions' value and efficiency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 32 of its 61 references that are in the map. First posted on arXiv 26 Oct 2025.

Builds on

32 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024RAGE Against the Machine: Retrieval-Augmented LLM Explanations11 May

Other references · 31

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  6. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  7. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  8. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 614
  9. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  10. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
  11. 2020Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Reviewcited by 174
  12. 2023Do Language Models Know When They're Hallucinating References?cited by 116
19 more
  1. 2024ContextCite: Attributing Model Generation to Contextcited by 98
  2. 2025Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samplescited by 72
  3. 2019Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QAcited by 63
  4. 2019Understanding Dataset Design Choices for Multi-hop Reasoningcited by 50
  5. 2024CiteME: Can Language Models Accurately Cite Scientific Claims?cited by 48
  6. 2020Decisions, Counterfactual Explanations and Strategic Behaviorcited by 44
  7. 2024LLM-Ensemble: Optimal Large Language Model Ensemble Method for E-commerce Product Attribute Value Extractioncited by 39
  8. 2020Citation Recommendation: Approaches and Datasetscited by 36
  9. 2024Source-Aware Training Enables Knowledge Attribution in Language Modelscited by 30
  10. 2024Citation-Enhanced Generation for LLM-based Chatbotscited by 25
  11. 2024InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniquescited by 25
  12. 2025Hard Negatives, Hard Lessons: Revisiting Training Data Quality for Robust Information Retrieval with LLMscited by 17
  13. 2024Verifiable Generation with Subsentence-Level Fine-Grained Citationscited by 15
  14. 2022BASED-XAI: Breaking Ablation Studies Down for Explainable Artificial Intelligencecited by 13
  15. 2024Learning to Generate Answers with Citations via Factual Consistency Modelscited by 13
  16. 2021Mitigating False-Negative Contexts in Multi-document Question Answering with Retrieval Marginalizationcited by 10
  17. 2025CiteFix: Enhancing RAG Accuracy Through Post-Processing Citation Correctioncited by 8
  18. 2015The Retrieval Effectiveness of Web Search Engines: Considering Results Descriptionscited by 4
  19. 2022Computing Rule-Based Explanations by Leveraging Counterfactualscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026RUBEN: Rule-Based Explanations for Retrieval-Augmented LLM Systems
  2. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  3. 2024Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspective
  4. 2025Improve Rule Retrieval and Reasoning with Self-Induction and Relevance ReEstimate
  5. 2024Let your LLM generate a few tokens and you will reduce the need for retrieval
  6. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Survey
  8. 2024On the Role of Long-tail Knowledge in Retrieval Augmented Large Language Models