# HalluTruthQA-4K: A Fine-Grained Corpus and Annotation Process for Arabic Hallucination Detection and Truth Verification

arXiv: https://arxiv.org/abs/2608.03966 · HTML: https://mlsanity.com/paper/2608.03966/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 29 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 22 of 22)

- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2305.11747/) (2023)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://mlsanity.com/paper/2401.00396/) (2023)
- [Fine-grained Hallucination Detection and Editing for Language Models](https://mlsanity.com/paper/2401.06855/) (2024)
- [Fanar: An Arabic-Centric Multimodal Generative AI Platform](https://mlsanity.com/paper/2501.13944/) (2025)
- [FELM: Benchmarking Factuality Evaluation of Large Language Models](https://mlsanity.com/paper/2310.00741/) (2023)
- [Large Language Models Hallucination: A Comprehensive Survey](https://mlsanity.com/paper/2510.06265/) (2025)
- [SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared Task on Hallucinations and Related Observable Overgeneration Mistakes](https://mlsanity.com/paper/2504.11975/) (2025)
- [PalmX 2025: The First Shared Task on Benchmarking LLMs on Arabic and Islamic Culture](https://mlsanity.com/paper/2509.02550/) (2025)
- [AraHalluEval: A Fine-grained Hallucination Evaluation Framework for Arabic LLMs](https://mlsanity.com/paper/2509.04656/) (2025)
- [Assessing Large Language Models on Islamic Legal Reasoning: Evidence from Inheritance Law Evaluation](https://mlsanity.com/paper/2509.01081/) (2025)
- [From RAG to Agentic RAG for Faithful Islamic Question Answering](https://mlsanity.com/paper/2601.07528/) (2026)
- [MAWARITH: A Dataset and Benchmark for Legal Inheritance Reasoning with LLMs](https://mlsanity.com/paper/2603.07539/) (2026)
- [IslamicMMLU: A Benchmark for Evaluating LLMs on Islamic Knowledge](https://mlsanity.com/paper/2603.23750/) (2026)
- [Span-Level Hallucination Detection for LLM-Generated Answers](https://mlsanity.com/paper/2504.18639/) (2025)
- [QIAS 2026: Overview of the Shared Task on Islamic Inheritance Reasoning](https://mlsanity.com/paper/2606.13756/) (2026)
- [Halluverse-M^3: A multitask multilingual benchmark for hallucination in LLMs](https://mlsanity.com/paper/2602.06920/) (2026)
- [HalluScore: Large Language Model Hallucination Question Answering Benchmark](https://mlsanity.com/paper/2605.17007/) (2026)
- [HalluTruthQA: A Fine-Grained Benchmark for Hallucination Detection, Localization, and Explanation in Arabic Question Answering](https://mlsanity.com/paper/2607.20219/) (2026)

## Built on by (most cited first, 1 of 1)

- [Halluscoring 2026: The first shared task on llms hallucination detection and answer verification](https://mlsanity.com/paper/2609.38355/) (2026)

## Abstract (arXiv)

Large language models can generate fluent Arabic answers while introducing factual errors that are difficult to identify and verify. Existing Arabic hallucination resources often assign a binary label to an entire response, indicating whether it is hallucinated or non-hallucinated, but provide limited information about the exact erroneous content, the reason for the error, or the correct factual answer. We present HalluTruthQA-4K, an expanded version of the HalluTruthQA resource containing 4,000 expert-curated Arabic question-answering instances across four knowledge-intensive domains: Islamic knowledge, history, science, and geography. Serving as the official dataset for Track 2 of the HalluScoring 2026 shared task, HalluTruthQA-4K extends our original corpus to 4,000 instances. Each instance pairs an Arabic question with a model-generated response, a verified reference answer, and five plausible distractors. Hallucinated responses are additionally annotated with character-level erroneous spans, human-written explanations, and hierarchical hallucination types. The corpus contains 1,643 hallucinated and 2,357 non-hallucinated responses, with 1,843 annotated erroneous spans. We describe the resource construction and annotation methodology, including question selection, controlled answer generation, candidate construction, expert annotation, independent verification, adjudication, and quality control. We also document the annotation guidelines, taxonomy, data format, inter-annotator agreement, and corpus statistics. HalluTruthQA-4K provides a reusable resource for hallucination detection, span-level error localization, explanation generation, factual verification, and the broader evaluation of factual reliability in Arabic language models.