# A Benchmark Study of Contrastive Learning for Arabic Social Meaning

arXiv: https://arxiv.org/abs/2210.12314 · HTML: https://mlsanity.com/paper/2210.12314/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 97 references that are in the map.
- First posted on arXiv 22 Oct 2022.

## Builds on (most cited first, 30 of 41)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Efficient Natural Language Response Suggestion for Smart Reply](https://mlsanity.com/paper/1705.00652/) (2017)
- [An efficient framework for learning sentence representations](https://mlsanity.com/paper/1803.02893/) (2018)
- [CERT: Contrastive Self-supervised Learning for Language Understanding](https://mlsanity.com/paper/2005.12766/) (2020)
- [ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic](https://mlsanity.com/paper/2101.01785/) (2020)
- [COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining](https://mlsanity.com/paper/2102.08473/) (2021)
- [Learning Dense Representations for Entity Retrieval](https://mlsanity.com/paper/1909.10506/) (2019)
- [The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://mlsanity.com/paper/2103.06678/) (2021)
- [On Sampling Strategies for Neural Network-based Collaborative Filtering](https://mlsanity.com/paper/1706.07881/) (2017)
- [AraT5: Text-to-Text Transformers for Arabic Language Generation](https://mlsanity.com/paper/2109.12068/) (2021)
- [Debiased Contrastive Learning of Unsupervised Sentence Representations](https://mlsanity.com/paper/2205.00656/) (2022)
- [NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2010.11334/) (2020)
- [Arabic Dialect Identification in the Wild](https://mlsanity.com/paper/2005.06557/) (2020)
- [Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification](https://mlsanity.com/paper/2109.05427/) (2021)
- [CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding](https://mlsanity.com/paper/2010.08670/) (2020)
- [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://mlsanity.com/paper/2102.10684/) (2021)
- [Improved Text Classification via Contrastive Adversarial Training](https://mlsanity.com/paper/2107.10137/) (2021)
- [NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2103.08466/) (2021)
- [TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning](https://mlsanity.com/paper/2111.04198/) (2021)
- [Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embedding](https://mlsanity.com/paper/2202.13093/) (2022)
- [Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identification](https://mlsanity.com/paper/1808.07674/) (2018)
- [SCD: Self-Contrastive Decorrelation for Sentence Embeddings](https://mlsanity.com/paper/2203.07847/) (2022)

## Built on by (most cited first, 2 of 2)

- [Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarks](https://mlsanity.com/paper/2411.01192/) (2024)
- [A comprehensive survey of contemporary Arabic sentiment analysis: Methods, Challenges, and Future Directions](https://mlsanity.com/paper/2502.03827/) (2025)

## Abstract (arXiv)

Contrastive learning (CL) brought significant progress to various NLP tasks. Despite this progress, CL has not been applied to Arabic NLP to date. Nor is it clear how much benefits it could bring to particular classes of tasks such as those involved in Arabic social meaning (e.g., sentiment analysis, dialect identification, hate speech detection). In this work, we present a comprehensive benchmark study of state-of-the-art supervised CL methods on a wide array of Arabic social meaning tasks. Through extensive empirical analyses, we show that CL methods outperform vanilla finetuning on most tasks we consider. We also show that CL can be data efficient and quantify this efficiency. Overall, our work allows us to demonstrate the promise of CL methods, including in low-resource settings.