# ArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD

arXiv: https://arxiv.org/abs/2205.09685 · HTML: https://mlsanity.com/paper/2205.09685/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 19 May 2022.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic](https://mlsanity.com/paper/2101.01785/) (2020)
- [The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://mlsanity.com/paper/2103.06678/) (2021)
- [GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge](https://mlsanity.com/paper/1908.07245/) (2019)
- [Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders](https://mlsanity.com/paper/2005.02590/) (2020)
- [Entity Linking in 100 Languages](https://mlsanity.com/paper/2011.02690/) (2020)
- [A Panoramic Survey of Natural Language Processing in the Arab World](https://mlsanity.com/paper/2011.12631/) (2020)
- [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://mlsanity.com/paper/2102.10684/) (2021)
- [Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences](https://mlsanity.com/paper/2009.11795/) (2020)
- [Extracting Synonyms from Bilingual Dictionaries](https://mlsanity.com/paper/2012.00600/) (2020)
- [Word Sense Disambiguation for 158 Languages using Word Embeddings Only](https://mlsanity.com/paper/2003.06651/) (2020)

## Built on by (most cited first, 18 of 18)

- [AraFinNLP 2024: The First Arabic Financial NLP Shared Task](https://mlsanity.com/paper/2407.09818/) (2024)
- [Nabra: Syrian Arabic Dialects with Morphological Annotations](https://mlsanity.com/paper/2310.17315/) (2023)
- [A Benchmark and Scoring Algorithm for Enriching Arabic Synonyms](https://mlsanity.com/paper/2302.02232/) (2023)
- [SALMA: Arabic Sense-Annotated Corpus and WSD Benchmarks](https://mlsanity.com/paper/2310.19029/) (2023)
- [Lisan: Yemeni, Iraqi, Libyan, and Sudanese Arabic Dialect Copora with Morphological Annotations](https://mlsanity.com/paper/2212.06468/) (2022)
- [ArBanking77: Intent Detection Neural Model and a New Dataset in Modern and Dialectical Arabic](https://mlsanity.com/paper/2310.19034/) (2023)
- [Arabic Fine-Grained Entity Recognition](https://mlsanity.com/paper/2310.17333/) (2023)
- [Context-Gloss Augmentation for Improving Arabic Target Sense Verification](https://mlsanity.com/paper/2302.03126/) (2023)
- [Offensive Hebrew Corpus and Detection using BERT](https://mlsanity.com/paper/2309.02724/) (2023)
- [WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task](https://mlsanity.com/paper/2407.09936/) (2024)
- [Qabas: An Open-Source Arabic Lexicographic Database](https://mlsanity.com/paper/2406.06598/) (2024)
- [NLU-STR at SemEval-2024 Task 1: Generative-based Augmentation and Encoder-based Scoring for Semantic Textual Relatedness](https://mlsanity.com/paper/2405.00659/) (2024)
- [Event-Arguments Extraction Corpus and Modeling using BERT for Arabic](https://mlsanity.com/paper/2407.21153/) (2024)
- [Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition](https://mlsanity.com/paper/2506.12615/) (2025)
- [ArabicNLU 2024: The First Arabic Natural Language Understanding Shared Task](https://mlsanity.com/paper/2407.20663/) (2024)
- [Enhancing Documents with Multidimensional Relevance Statements in Cross-encoder Re-ranking](https://mlsanity.com/paper/2306.10979/) (2023)
- [QuranMorph: Morphologically Annotated Quranic Corpus](https://mlsanity.com/paper/2506.18148/) (2025)
- [SinaTools: Open Source Toolkit for Arabic Natural Language Processing](https://mlsanity.com/paper/2411.01523/) (2024)

## Abstract (arXiv)

Using pre-trained transformer models such as BERT has proven to be effective in many NLP tasks. This paper presents our work to fine-tune BERT models for Arabic Word Sense Disambiguation (WSD). We treated the WSD task as a sentence-pair binary classification task. First, we constructed a dataset of labeled Arabic context-gloss pairs (~167k pairs) we extracted from the Arabic Ontology and the large lexicographic database available at Birzeit University. Each pair was labeled as True or False and target words in each context were identified and annotated. Second, we used this dataset for fine-tuning three pre-trained Arabic BERT models. Third, we experimented the use of different supervised signals used to emphasize target words in context. Our experiments achieved promising results (accuracy of 84%) although we used a large set of senses in the experiment.