# SALMA: Arabic Sense-Annotated Corpus and WSD Benchmarks

arXiv: https://arxiv.org/abs/2310.19029 · HTML: https://mlsanity.com/paper/2310.19029/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 52 references that are in the map.
- First posted on arXiv 29 Oct 2023.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge](https://mlsanity.com/paper/1908.07245/) (2019)
- [Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders](https://mlsanity.com/paper/2005.02590/) (2020)
- [A Panoramic Survey of Natural Language Processing in the Arab World](https://mlsanity.com/paper/2011.12631/) (2020)
- [Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT](https://mlsanity.com/paper/2205.09651/) (2022)
- [Curras + Baladi: Towards a Levantine Corpus](https://mlsanity.com/paper/2205.09692/) (2022)
- [The Arabic Ontology -- An Arabic Wordnet with Ontologically Clean Content](https://mlsanity.com/paper/2205.09664/) (2022)
- [ArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD](https://mlsanity.com/paper/2205.09685/) (2022)
- [Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences](https://mlsanity.com/paper/2009.11795/) (2020)
- [WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context](https://mlsanity.com/paper/2004.15016/) (2020)
- [Extracting Synonyms from Bilingual Dictionaries](https://mlsanity.com/paper/2012.00600/) (2020)
- [Nabra: Syrian Arabic Dialects with Morphological Annotations](https://mlsanity.com/paper/2310.17315/) (2023)
- [A Benchmark and Scoring Algorithm for Enriching Arabic Synonyms](https://mlsanity.com/paper/2302.02232/) (2023)
- [Lisan: Yemeni, Iraqi, Libyan, and Sudanese Arabic Dialect Copora with Morphological Annotations](https://mlsanity.com/paper/2212.06468/) (2022)
- [Context-Gloss Augmentation for Improving Arabic Target Sense Verification](https://mlsanity.com/paper/2302.03126/) (2023)
- [Current Trends and Approaches in Synonyms Extraction: Potential Adaptation to Arabic](https://mlsanity.com/paper/2205.10412/) (2022)

## Built on by (most cited first, 10 of 10)

- [AraFinNLP 2024: The First Arabic Financial NLP Shared Task](https://mlsanity.com/paper/2407.09818/) (2024)
- [WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task](https://mlsanity.com/paper/2407.09936/) (2024)
- [Qabas: An Open-Source Arabic Lexicographic Database](https://mlsanity.com/paper/2406.06598/) (2024)
- [NLU-STR at SemEval-2024 Task 1: Generative-based Augmentation and Encoder-based Scoring for Semantic Textual Relatedness](https://mlsanity.com/paper/2405.00659/) (2024)
- [Event-Arguments Extraction Corpus and Modeling using BERT for Arabic](https://mlsanity.com/paper/2407.21153/) (2024)
- [Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition](https://mlsanity.com/paper/2506.12615/) (2025)
- [ArabicNLU 2024: The First Arabic Natural Language Understanding Shared Task](https://mlsanity.com/paper/2407.20663/) (2024)
- [QuranMorph: Morphologically Annotated Quranic Corpus](https://mlsanity.com/paper/2506.18148/) (2025)
- [SinaTools: Open Source Toolkit for Arabic Natural Language Processing](https://mlsanity.com/paper/2411.01523/) (2024)
- [QuranicMMLU: A Cognitively-Aware Benchmark for Evaluating Generative AI Solutions on Quranic Linguistic Knowledge](https://mlsanity.com/paper/2609.22038/) (2026)

## Abstract (arXiv)

SALMA, the first Arabic sense-annotated corpus, consists of ~34K tokens, which are all sense-annotated. The corpus is annotated using two different sense inventories simultaneously (Modern and Ghani). SALMA novelty lies in how tokens and senses are associated. Instead of linking a token to only one intended sense, SALMA links a token to multiple senses and provides a score to each sense. A smart web-based annotation tool was developed to support scoring multiple senses against a given word. In addition to sense annotations, we also annotated the corpus using six types of named entities. The quality of our annotations was assessed using various metrics (Kappa, Linear Weighted Kappa, Quadratic Weighted Kappa, Mean Average Error, and Root Mean Square Error), which show very high inter-annotator agreement. To establish a Word Sense Disambiguation baseline using our SALMA corpus, we developed an end-to-end Word Sense Disambiguation system using Target Sense Verification. We used this system to evaluate three Target Sense Verification models available in the literature. Our best model achieved an accuracy with 84.2% using Modern and 78.7% using Ghani. The full corpus and the annotation tool are open-source and publicly available at https://sina.birzeit.edu/salma/.