# RUBEN: Rule-Based Explanations for Retrieval-Augmented LLM Systems

arXiv: https://arxiv.org/abs/2605.10862 · HTML: https://mlsanity.com/paper/2605.10862/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 32 of its 61 references that are in the map.
- First posted on arXiv 11 May 2026.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)
- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 30 of 32)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review](https://mlsanity.com/paper/2010.10596/) (2020)
- [Do Language Models Know When They're Hallucinating References?](https://mlsanity.com/paper/2305.18248/) (2023)
- [ContextCite: Attributing Model Generation to Context](https://mlsanity.com/paper/2409.00729/) (2024)
- [Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples](https://mlsanity.com/paper/2510.07192/) (2025)
- [Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA](https://mlsanity.com/paper/1906.07132/) (2019)
- [Understanding Dataset Design Choices for Multi-hop Reasoning](https://mlsanity.com/paper/1904.12106/) (2019)
- [CiteME: Can Language Models Accurately Cite Scientific Claims?](https://mlsanity.com/paper/2407.12861/) (2024)
- [Decisions, Counterfactual Explanations and Strategic Behavior](https://mlsanity.com/paper/2002.04333/) (2020)
- [LLM-Ensemble: Optimal Large Language Model Ensemble Method for E-commerce Product Attribute Value Extraction](https://mlsanity.com/paper/2403.00863/) (2024)
- [Citation Recommendation: Approaches and Datasets](https://mlsanity.com/paper/2002.06961/) (2020)
- [Source-Aware Training Enables Knowledge Attribution in Language Models](https://mlsanity.com/paper/2404.01019/) (2024)
- [Citation-Enhanced Generation for LLM-based Chatbots](https://mlsanity.com/paper/2402.16063/) (2024)
- [InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniques](https://mlsanity.com/paper/2407.14494/) (2024)
- [Hard Negatives, Hard Lessons: Revisiting Training Data Quality for Robust Information Retrieval with LLMs](https://mlsanity.com/paper/2505.16967/) (2025)
- [Verifiable Generation with Subsentence-Level Fine-Grained Citations](https://mlsanity.com/paper/2406.06125/) (2024)
- [BASED-XAI: Breaking Ablation Studies Down for Explainable Artificial Intelligence](https://mlsanity.com/paper/2207.05566/) (2022)
- [Learning to Generate Answers with Citations via Factual Consistency Models](https://mlsanity.com/paper/2406.13124/) (2024)
- [Mitigating False-Negative Contexts in Multi-document Question Answering with Retrieval Marginalization](https://mlsanity.com/paper/2103.12235/) (2021)
- [RAGE Against the Machine: Retrieval-Augmented LLM Explanations](https://mlsanity.com/paper/2405.13000/) (2024)
- [CiteFix: Enhancing RAG Accuracy Through Post-Processing Citation Correction](https://mlsanity.com/paper/2504.15629/) (2025)

## Built on by (most cited first, 1 of 1)

- [ExecRubrics: Executable Tool-Augmented Rubrics for Verifiable and Efficient Long-Form Evaluation](https://mlsanity.com/paper/2608.22559/) (2026)

## Abstract (arXiv)

This paper demonstrates RUBEN, an interactive tool for discovering minimal rules to explain the outputs of retrieval-augmented large language models (LLMs) in data-driven applications. We leverage novel pruning strategies to efficiently identify a minimal set of rules that subsume all others. We further demonstrate novel applications of these rules for LLM safety, specifically to test the resiliency of safety training and effectiveness of adversarial prompt injections.