# From RAG to Agentic RAG for Faithful Islamic Question Answering

arXiv: https://arxiv.org/abs/2601.07528 · HTML: https://mlsanity.com/paper/2601.07528/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 32 of its 113 references that are in the map.
- First posted on arXiv 12 Jan 2026.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 32)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Fanar: An Arabic-Centric Multimodal Generative AI Platform](https://mlsanity.com/paper/2501.13944/) (2025)
- [SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge](https://mlsanity.com/paper/2509.07968/) (2025)
- [FIRST: A Framework for Optimizing Information Quality in Mobile Crowdsensing Systems](https://mlsanity.com/paper/1804.11147/) (2018)
- [Inverse Scaling in Test-Time Compute](https://mlsanity.com/paper/2507.14417/) (2025)
- [Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in](https://mlsanity.com/paper/2404.18460/) (2024)
- [Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs](https://mlsanity.com/paper/2507.09477/) (2025)
- [SeaLLMs 3: Open Foundation and Chat Multilingual Large Language Models for Southeast Asian Languages](https://mlsanity.com/paper/2407.19672/) (2024)
- [How Reliable is Multilingual LLM-as-a-Judge?](https://mlsanity.com/paper/2505.12201/) (2025)
- [On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena](https://mlsanity.com/paper/2501.04662/) (2025)
- [LLM Ethics Benchmark: A Three-Dimensional Assessment System for Evaluating Moral Reasoning in Large Language Models](https://mlsanity.com/paper/2505.00853/) (2025)
- [PalmX 2025: The First Shared Task on Benchmarking LLMs on Arabic and Islamic Culture](https://mlsanity.com/paper/2509.02550/) (2025)
- [Reasoning RAG via System 1 or System 2: A Survey on Reasoning Agentic Retrieval-Augmented Generation for Industry Challenges](https://mlsanity.com/paper/2506.10408/) (2025)
- [CARE: Multilingual Human Preference Learning for Cultural Awareness](https://mlsanity.com/paper/2504.05154/) (2025)
- [The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technology](https://mlsanity.com/paper/2506.01340/) (2025)
- [AraHalluEval: A Fine-grained Hallucination Evaluation Framework for Arabic LLMs](https://mlsanity.com/paper/2509.04656/) (2025)
- [Test-Time Scaling in Reasoning Models Is Not Effective for Knowledge-Intensive Tasks Yet](https://mlsanity.com/paper/2509.06861/) (2025)
- [Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarks](https://mlsanity.com/paper/2411.01192/) (2024)
- [Sacred or Synthetic? Evaluating LLM Reliability and Abstention for Religious Questions](https://mlsanity.com/paper/2508.08287/) (2025)
- [Assessing Large Language Models on Islamic Legal Reasoning: Evidence from Inheritance Law Evaluation](https://mlsanity.com/paper/2509.01081/) (2025)
- [MedEthicEval: Evaluating Large Language Models Based on Chinese Medical Ethics](https://mlsanity.com/paper/2503.02374/) (2025)
- [MedEthicsQA: A Comprehensive Question Answering Benchmark for Medical Ethics Evaluation of LLMs](https://mlsanity.com/paper/2506.22808/) (2025)
- [TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law](https://mlsanity.com/paper/2507.21134/) (2025)
- [Alignment at Pre-training! Towards Native Alignment for Arabic LLMs](https://mlsanity.com/paper/2412.03253/) (2024)
- [Span-Level Hallucination Detection for LLM-Generated Answers](https://mlsanity.com/paper/2504.18639/) (2025)
- [DTW at Qur'an QA 2022: Utilising Transfer Learning with Transformers for Question Answering in a Low-resource Domain](https://mlsanity.com/paper/2205.06025/) (2022)
- [Quranic Audio Dataset: Crowdsourced and Labeled Recitation from Non-Arabic Speakers](https://mlsanity.com/paper/2405.02675/) (2024)
- [Prompt Engineering Techniques for Mitigating Cultural Bias Against Arabs and Muslims in Large Language Models: A Systematic Review](https://mlsanity.com/paper/2506.18199/) (2025)
- [When Large Language Models Meet Law: Dual-Lens Taxonomy, Technical Advances, and Ethical Governance](https://mlsanity.com/paper/2507.07748/) (2025)

## Built on by (most cited first, 9 of 9)

- [MAWARITH: A Dataset and Benchmark for Legal Inheritance Reasoning with LLMs](https://mlsanity.com/paper/2603.07539/) (2026)
- [QIAS 2026: Overview of the Shared Task on Islamic Inheritance Reasoning](https://mlsanity.com/paper/2606.13756/) (2026)
- [Fanar-Sadiq: A Multi-Agent Architecture for Grounded Islamic QA](https://mlsanity.com/paper/2603.08501/) (2026)
- [QU-NLP at QIAS 2026: Multi-Stage QLoRA Fine-Tuning for Arabic Islamic Inheritance Reasoning](https://mlsanity.com/paper/2604.16396/) (2026)
- [HalluTruthQA-4K: A Fine-Grained Corpus and Annotation Process for Arabic Hallucination Detection and Truth Verification](https://mlsanity.com/paper/2608.03966/) (2026)
- [Detecting Hallucinations and Recovering Verified Answers in Arabic Islamic Question Answering](https://mlsanity.com/paper/2608.03720/) (2026)
- [IslamicTurathBench: A Multi-Task, Multi-Discipline Benchmark for Evaluating Large Language Models on the Islamic Scholarly Tradition (turath)](https://mlsanity.com/paper/2608.04703/) (2026)
- [What Makes a Good Fiqh Retriever? Answer Retrieval for Arabic Islamic Jurisprudence](https://mlsanity.com/paper/2608.20246/) (2026)
- [When Do VLMs Help Arabic Manuscript OCR? A Cross-Dataset Study](https://mlsanity.com/paper/2608.22366/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly used for Islamic question answering, where ungrounded responses may carry serious religious consequences. Yet standard MCQ/MRC-style evaluations (MCQ: Multiple choice questions, MRC: Machine Reading Comprehension) do not capture key real-world failure modes, notably free-form hallucinations and the ability to abstain when evidence is insufficient. To address this gap, we introduce IslamicFaithQA, a 3,810-item bilingual (Arabic/English) generative benchmark with atomic single-gold answers, which enables direct measurement of hallucination and abstention. We additionally developed an end-to-end grounded Islamic modeling suite consisting of (i) 25K Arabic text-grounded SFT reasoning pairs, (ii) 5K bilingual preference samples for reward-guided alignment, and (iii) a verse-level Qur'an retrieval corpus of ~6k atomic verses (ayat). Building on these resources, we develop an agentic Quran-grounding framework (agentic RAG) that uses structured tool calls for iterative evidence seeking and answer revision. Experiments across Arabic-centric and multilingual LLMs show that retrieval improves correctness and that agentic RAG yields the largest gains beyond standard RAG, achieving state-of-the-art performance and stronger Arabic-English robustness even with a small model (i.e., Qwen3 4B). We made the datasets are publicly available. https://huggingface.co/datasets/QCRI/IslamicFaithQA