# ArabicaQA: A Comprehensive Dataset for Arabic Question Answering

arXiv: https://arxiv.org/abs/2403.17848 · HTML: https://mlsanity.com/paper/2403.17848/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 63 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [FastText.zip: Compressing text classification models](https://mlsanity.com/paper/1612.03651/) (2016)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension](https://mlsanity.com/paper/1810.12885/) (2018)
- [Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering](https://mlsanity.com/paper/2307.16877/) (2023)
- [Neural Arabic Question Answering](https://mlsanity.com/paper/1906.05394/) (2019)
- [A Panoramic Survey of Natural Language Processing in the Arab World](https://mlsanity.com/paper/2011.12631/) (2020)
- [ArchivalQA: A Large-scale Benchmark Dataset for Open Domain Question Answering over Historical News Collections](https://mlsanity.com/paper/2109.03438/) (2021)
- [Towards Best Practices for Training Multilingual Dense Retrieval Models](https://mlsanity.com/paper/2204.02363/) (2022)
- [Who did What: A Large-Scale Person-Centered Cloze Dataset](https://mlsanity.com/paper/1608.05457/) (2016)
- [Automated Question Answer medical model based on Deep Learning Technology](https://mlsanity.com/paper/2005.10416/) (2020)
- [An open access NLP dataset for Arabic dialects : Data collection, labeling, and model construction](https://mlsanity.com/paper/2102.11000/) (2021)

## Built on by (most cited first, 20 of 20)

- [CAMEL-Bench: A Comprehensive Arabic LMM Benchmark](https://mlsanity.com/paper/2410.18976/) (2024)
- [Command R7B Arabic: A Small, Enterprise Focused, Multilingual, and Culturally Aware Arabic LLM](https://mlsanity.com/paper/2503.14603/) (2025)
- [Rankify: A Comprehensive Python Toolkit for Retrieval, Re-Ranking, and Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.02464/) (2025)
- [ASRank: Zero-Shot Re-Ranking with Answer Scent for Document Retrieval](https://mlsanity.com/paper/2501.15245/) (2025)
- [From Guidelines to Practice: A New Paradigm for Arabic Language Model Evaluation](https://mlsanity.com/paper/2506.01920/) (2025)
- [101 Billion Arabic Words Dataset](https://mlsanity.com/paper/2405.01590/) (2024)
- [GemmAr: Enhancing LLMs Through Arabic Instruction-Tuning](https://mlsanity.com/paper/2407.02147/) (2024)
- [DynRank: Improving Passage Retrieval with Dynamic Zero-Shot Prompting Based on Question Classification](https://mlsanity.com/paper/2412.00600/) (2024)
- [AraTable: Benchmarking LLMs' Reasoning and Understanding of Arabic Tabular Data](https://mlsanity.com/paper/2507.18442/) (2025)
- [Two-Stage Quranic QA via Ensemble Retrieval and Instruction-Tuned Answer Extraction](https://mlsanity.com/paper/2508.06971/) (2025)
- [From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledge](https://mlsanity.com/paper/2510.20043/) (2025)
- [A Comprehensive Survey of Masked Faces: Recognition, Detection, and Unmasking](https://mlsanity.com/paper/2405.05900/) (2024)
- [Enhanced Arabic Text Retrieval with Attentive Relevance Scoring](https://mlsanity.com/paper/2507.23404/) (2025)
- [Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challenges](https://mlsanity.com/paper/2608.17605/) (2026)
- [AutoArabic: A Three-Stage Framework for Localizing Video-Text Retrieval Benchmarks](https://mlsanity.com/paper/2509.16438/) (2025)
- [Transformer Tafsir at QIAS 2025 Shared Task: Hybrid Retrieval-Augmented Generation for Islamic Knowledge Question Answering](https://mlsanity.com/paper/2509.23793/) (2025)
- [The Role of Dataset Linguistic Structure in the Cultural Awareness of Large Language Models](https://mlsanity.com/paper/2602.01161/) (2026)
- [BETA-Labeling for Multilingual Dataset Construction in Low-Resource IR](https://mlsanity.com/paper/2602.14488/) (2026)
- [BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment](https://mlsanity.com/paper/2604.07201/) (2026)
- [Large Language Models Systematically Favor Popular Options: Evidence and Mitigation Across MCQs](https://mlsanity.com/paper/2608.29257/) (2026)

## Abstract (arXiv)

In this paper, we address the significant gap in Arabic natural language processing (NLP) resources by introducing ArabicaQA, the first large-scale dataset for machine reading comprehension and open-domain question answering in Arabic. This comprehensive dataset, consisting of 89,095 answerable and 3,701 unanswerable questions created by crowdworkers to look similar to answerable ones, along with additional labels of open-domain questions marks a crucial advancement in Arabic NLP resources. We also present AraDPR, the first dense passage retrieval model trained on the Arabic Wikipedia corpus, specifically designed to tackle the unique challenges of Arabic text retrieval. Furthermore, our study includes extensive benchmarking of large language models (LLMs) for Arabic question answering, critically evaluating their performance in the Arabic language context. In conclusion, ArabicaQA, AraDPR, and the benchmarking of LLMs in Arabic question answering offer significant advancements in the field of Arabic NLP. The dataset and code are publicly accessible for further research https://github.com/DataScienceUIBK/ArabicaQA.