ArabicNLU 2024: The First Arabic Natural Language Understanding Shared Task
Not on the map. Why?
Abstract (arXiv)
This paper presents an overview of the Arabic Natural Language Understanding (ArabicNLU 2024) shared task, focusing on two subtasks: Word Sense Disambiguation (WSD) and Location Mention Disambiguation (LMD). The task aimed to evaluate the ability of automated systems to resolve word ambiguity and identify locations mentioned in Arabic text. We provided participants with novel datasets, including a sense-annotated corpus for WSD, called SALMA with approximately 34k annotated tokens, and the IDRISI-DA dataset with 3,893 annotations and 763 unique location mentions. These are challenging tasks. Out of the 38 registered teams, only three teams participated in the final evaluation phase, with the highest accuracy being 77.8% for WSD and the highest MRR@1 being 95.0% for LMD. The shared task not only facilitated the evaluation and comparison of different techniques, but also provided valuable insights and resources for the continued advancement of Arabic NLU technologies.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 62 references that are in the map. First posted on arXiv 30 Jul 2024.
Builds on
23 of 62 references.
References · 23
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
- 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 359
- 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 250
- 2020ARBERT & MARBERT: Deep Bidirectional Transformers for Arabiccited by 164
- 2023AceGPT, Localizing Large Language Models in Arabiccited by 75
- 2022Wojood: Nested Arabic Named Entity Corpus and Recognition using BERTcited by 32
- 2023NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Taskcited by 29
- 2022Curras + Baladi: Towards a Levantine Corpuscited by 20
- 2022The Arabic Ontology -- An Arabic Wordnet with Ontologically Clean Contentcited by 19
- 2022ArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSDcited by 18
- 2024Are Large Language Models the New Interface for Data Pipelines?cited by 12
11 more
- 2024AraFinNLP 2024: The First Arabic Financial NLP Shared Taskcited by 12
- 2023SALMA: Arabic Sense-Annotated Corpus and WSD Benchmarkscited by 10
- 2021LU-BZU at SemEval-2021 Task 2: Word2Vec and Lemma2Vec performance in Arabic Word-in-Context disambiguationcited by 9
- 2023ArBanking77: Intent Detection Neural Model and a New Dataset in Modern and Dialectical Arabiccited by 8
- 2023Context-Gloss Augmentation for Improving Arabic Target Sense Verificationcited by 6
- 2023WojoodNER 2023: The First Arabic Named Entity Recognition Shared Taskcited by 6
- 2024WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Taskcited by 6
- 2019DLocRL: A Deep Learning Pipeline for Fine-Grained Location Recognition and Linking in Tweetscited by 5
- 2024A Survey on Lexical Ambiguity Detection and Word Sense Disambiguationcited by 5
- 2020Are We There Yet? Evaluating State-of-the-Art Neural Network based Geoparsers Using EUPEG as a Benchmarking Platformcited by 4
- 2024Qabas: An Open-Source Arabic Lexicographic Databasecited by 4
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.