# AraHealthQA 2025: The First Shared Task on Arabic Health Question Answering

arXiv: https://arxiv.org/abs/2508.20047 · HTML: https://mlsanity.com/paper/2508.20047/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 27 Aug 2025.

## Builds on (most cited first, 11 of 11)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Support](https://mlsanity.com/paper/2106.01702/) (2021)
- [Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking Platform](https://mlsanity.com/paper/2110.05802/) (2021)
- [CAMS: An Annotated Corpus for Causal Analysis of Mental Health Issues in Social Media Posts](https://mlsanity.com/paper/2207.04674/) (2022)
- [MedArabiQ: Benchmarking Large Language Models on Arabic Medical Tasks](https://mlsanity.com/paper/2505.03427/) (2025)
- [AraSTEM: A Native Arabic Multiple Choice Question Benchmark for Evaluating LLMs Knowledge In STEM Subjects](https://mlsanity.com/paper/2501.00559/) (2024)
- [LOST: A Mental Health Dataset of Low Self-esteem in Reddit Posts](https://mlsanity.com/paper/2306.05596/) (2023)
- [MentalQA: An Annotated Arabic Corpus for Questions and Answers of Mental Healthcare](https://mlsanity.com/paper/2405.12619/) (2024)

## Built on by (most cited first, 2 of 2)

- [CARMA: Comprehensive Automatically-annotated Reddit Mental Health Dataset for Arabic](https://mlsanity.com/paper/2511.03102/) (2025)
- [How Correct Is Your Answer? A Semantic Correctness Framework for Open QA Evaluation](https://mlsanity.com/paper/2609.01369/) (2026)

## Abstract (arXiv)

We introduce AraHealthQA 2025, the Comprehensive Arabic Health Question Answering Shared Task, held in conjunction with ArabicNLP 2025 (co-located with EMNLP 2025). This shared task addresses the paucity of high-quality Arabic medical QA resources by offering two complementary tracks: MentalQA, focusing on Arabic mental health Q&A (e.g., anxiety, depression, stigma reduction), and MedArabiQ, covering broader medical domains such as internal medicine, pediatrics, and clinical decision making. Each track comprises multiple subtasks, evaluation datasets, and standardized metrics, facilitating fair benchmarking. The task was structured to promote modeling under realistic, multilingual, and culturally nuanced healthcare contexts. We outline the dataset creation, task design and evaluation framework, participation statistics, baseline systems, and summarize the overall outcomes. We conclude with reflections on the performance trends observed and prospects for future iterations in Arabic health QA.