Detecting Hallucinations and Recovering Verified Answers in Arabic Islamic Question Answering
Not on the map. Why?
Abstract (arXiv)
Large language models can generate fluent responses to Islamic questions while introducing factual errors that are difficult to identify. This paper presents our system for textscHalluScoring 2026 Task 2.1, textitIslamic Hallucination Detection and Find the Truth. The task requires a unified two-step prediction: determining whether an Arabic answer generated by an LLM is hallucinated and selecting the verified answer from six closely related candidate options. We use the Islamic knowledge dataset provided by the shared task, which contains 600 question--answer instances, including 341 hallucinated and 259 non-hallucinated answers. Our system is based on the fine-tuned textttgoogle/gemma-4-12B-it model and uses deterministic decoding during inference. The generated outputs are normalized to extract the hallucination label and the selected option. The system achieves a Macro-F1 score of 0.928 and a label accuracy of 0.935 for hallucination detection, together with an option accuracy of 0.895 for answer selection. These results yield a combined score of 0.912, demonstrating strong performance across both stages of the task. The lower option-selection accuracy indicates that distinguishing the verified answer from plausible alternatives remains more challenging than detecting hallucinated responses.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 9 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
6 of 9 references.
References · 6
- 2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
- 2025PalmX 2025: The First Shared Task on Benchmarking LLMs on Arabic and Islamic Culturecited by 21
- 2025Assessing Large Language Models on Islamic Legal Reasoning: Evidence from Inheritance Law Evaluationcited by 10
- 2026From RAG to Agentic RAG for Faithful Islamic Question Answeringcited by 9
- 2026MAWARITH: A Dataset and Benchmark for Legal Inheritance Reasoning with LLMscited by 8
- 2026HalluTruthQA: A Fine-Grained Benchmark for Hallucination Detection, Localization, and Explanation in Arabic Question Answeringcited by 2
Built on by
Not cited yet.