A Benchmark Study of Contrastive Learning for Arabic Social Meaning
Not on the map. Why?
Abstract (arXiv)
Contrastive learning (CL) brought significant progress to various NLP tasks. Despite this progress, CL has not been applied to Arabic NLP to date. Nor is it clear how much benefits it could bring to particular classes of tasks such as those involved in Arabic social meaning (e.g., sentiment analysis, dialect identification, hate speech detection). In this work, we present a comprehensive benchmark study of state-of-the-art supervised CL methods on a wide array of Arabic social meaning tasks. Through extensive empirical analyses, we show that CL methods outperform vanilla finetuning on most tasks we consider. We also show that CL can be data efficient and quantify this efficiency. Overall, our work allows us to demonstrate the promise of CL methods, including in low-resource settings.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 41 of its 97 references that are in the map. First posted on arXiv 22 Oct 2022.
Builds on
41 of 97 references.
References · 41
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2020Supervised Contrastive Learningcited by 2,829
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
- 2017Efficient Natural Language Response Suggestion for Smart Replycited by 229
- 2018An efficient framework for learning sentence representationscited by 198
- 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
29 more
- 2020ARBERT & MARBERT: Deep Bidirectional Transformers for Arabiccited by 166
- 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
- 2019Learning Dense Representations for Entity Retrievalcited by 122
- 2021The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Modelscited by 119
- 2017On Sampling Strategies for Neural Network-based Collaborative Filteringcited by 71
- 2021AraT5: Text-to-Text Transformers for Arabic Language Generationcited by 63
- 2022Debiased Contrastive Learning of Unsupervised Sentence Representationscited by 56
- 2020NADI 2020: The First Nuanced Arabic Dialect Identification Shared Taskcited by 47
- 2020Arabic Dialect Identification in the Wildcited by 45
- 2021Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classificationcited by 43
- 2020CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understandingcited by 41
- 2021Pre-Training BERT on Arabic Tweets: Practical Considerationscited by 39
- 2021Improved Text Classification via Contrastive Adversarial Trainingcited by 38
- 2021NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Taskcited by 32
- 2021TaCL: Improving BERT Pre-training with Token-aware Contrastive Learningcited by 32
- 2022Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embeddingcited by 25
- 2018Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identificationcited by 23
- 2022SCD: Self-Contrastive Decorrelation for Sentence Embeddingscited by 20
- 2019AraNet: A Deep Learning Toolkit for Arabic Social Mediacited by 14
- 2022Emojis as Anchors to Detect Arabic Offensive Language and Hate Speechcited by 14
- 2020Understanding and Detecting Dangerous Speech in Social Mediacited by 12
- 2020TUNIZI: a Tunisian Arabizi sentiment analysis Datasetcited by 11
- 2020Leveraging Affective Bidirectional Transformers for Offensive Language Detectioncited by 11
- 2021Improving Social Meaning Detection with Pragmatic Masking and Surrogate Fine-Tuningcited by 5
- 2021Improving Gradient-based Adversarial Training for Text Classification by Contrastive Learning and Auto-Encodercited by 4
- 2022Decay No More: A Persistent Twitter Dataset for Learning Social Meaningcited by 3
- 2022AlexU-AIC at Arabic Hate Speech 2022: Contrast to Classifycited by 3
- 2022ArabGend: Gender Analysis and Inference on Arabic Twittercited by 2
- 2022Meta AI at Arabic Hate Speech 2022: MultiTask Learning with Self-Correction for Hate Speech Classificationcited by 2
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.