# ArabicNLU 2024: The First Arabic Natural Language Understanding Shared Task

arXiv: https://arxiv.org/abs/2407.20663 · HTML: https://mlsanity.com/paper/2407.20663/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 62 references that are in the map.
- First posted on arXiv 30 Jul 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://mlsanity.com/paper/2309.11235/) (2023)
- [ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic](https://mlsanity.com/paper/2101.01785/) (2020)
- [AceGPT, Localizing Large Language Models in Arabic](https://mlsanity.com/paper/2309.12053/) (2023)
- [Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT](https://mlsanity.com/paper/2205.09651/) (2022)
- [NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2310.16117/) (2023)
- [Curras + Baladi: Towards a Levantine Corpus](https://mlsanity.com/paper/2205.09692/) (2022)
- [The Arabic Ontology -- An Arabic Wordnet with Ontologically Clean Content](https://mlsanity.com/paper/2205.09664/) (2022)
- [ArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD](https://mlsanity.com/paper/2205.09685/) (2022)
- [Are Large Language Models the New Interface for Data Pipelines?](https://mlsanity.com/paper/2406.06596/) (2024)
- [AraFinNLP 2024: The First Arabic Financial NLP Shared Task](https://mlsanity.com/paper/2407.09818/) (2024)
- [SALMA: Arabic Sense-Annotated Corpus and WSD Benchmarks](https://mlsanity.com/paper/2310.19029/) (2023)
- [LU-BZU at SemEval-2021 Task 2: Word2Vec and Lemma2Vec performance in Arabic Word-in-Context disambiguation](https://mlsanity.com/paper/2104.08110/) (2021)
- [ArBanking77: Intent Detection Neural Model and a New Dataset in Modern and Dialectical Arabic](https://mlsanity.com/paper/2310.19034/) (2023)
- [Context-Gloss Augmentation for Improving Arabic Target Sense Verification](https://mlsanity.com/paper/2302.03126/) (2023)
- [WojoodNER 2023: The First Arabic Named Entity Recognition Shared Task](https://mlsanity.com/paper/2310.16153/) (2023)
- [WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task](https://mlsanity.com/paper/2407.09936/) (2024)
- [DLocRL: A Deep Learning Pipeline for Fine-Grained Location Recognition and Linking in Tweets](https://mlsanity.com/paper/1901.07005/) (2019)
- [A Survey on Lexical Ambiguity Detection and Word Sense Disambiguation](https://mlsanity.com/paper/2403.16129/) (2024)
- [Are We There Yet? Evaluating State-of-the-Art Neural Network based Geoparsers Using EUPEG as a Benchmarking Platform](https://mlsanity.com/paper/2007.07455/) (2020)
- [Qabas: An Open-Source Arabic Lexicographic Database](https://mlsanity.com/paper/2406.06598/) (2024)

## Built on by (most cited first, 2 of 2)

- [Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition](https://mlsanity.com/paper/2506.12615/) (2025)
- [SinaTools: Open Source Toolkit for Arabic Natural Language Processing](https://mlsanity.com/paper/2411.01523/) (2024)

## Abstract (arXiv)

This paper presents an overview of the Arabic Natural Language Understanding (ArabicNLU 2024) shared task, focusing on two subtasks: Word Sense Disambiguation (WSD) and Location Mention Disambiguation (LMD). The task aimed to evaluate the ability of automated systems to resolve word ambiguity and identify locations mentioned in Arabic text. We provided participants with novel datasets, including a sense-annotated corpus for WSD, called SALMA with approximately 34k annotated tokens, and the IDRISI-DA dataset with 3,893 annotations and 763 unique location mentions. These are challenging tasks. Out of the 38 registered teams, only three teams participated in the final evaluation phase, with the highest accuracy being 77.8% for WSD and the highest MRR@1 being 95.0% for LMD. The shared task not only facilitated the evaluation and comparison of different techniques, but also provided valuable insights and resources for the continued advancement of Arabic NLU technologies.