Contrastive Learning of Sociopragmatic Meaning in Social Media
Not on the map. Why?
Abstract (arXiv)
Recent progress in representation and contrastive learning in NLP has not widely considered the class of textitsociopragmatic meaning (i.e., meaning in interaction within different language communities). To bridge this gap, we propose a novel framework for learning task-agnostic representations transferable to a wide range of sociopragmatic tasks (e.g., emotion, hate speech, humor, sarcasm). Our framework outperforms other contrastive learning frameworks for both in-domain and out-of-domain data, across both the general and few-shot settings. For example, compared to two popular pre-trained language models, our method obtains an improvement of 11.66 average F1 on 16 datasets when fine-tuned on only 20 training samples per dataset.Our code is available at: https://github.com/UBC-NLP/infodcl
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 41 of its 120 references that are in the map. First posted on arXiv 15 Mar 2022.
Builds on
41 of 120 references.
References · 41
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2020Supervised Contrastive Learningcited by 2,803
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
- 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
- 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
- 2020Contrastive Learning with Hard Negative Samplescited by 619
- 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
- 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
29 more
- 2020Understanding the Behaviour of Contrastive Losscited by 397
- 2020BERTweet: A pre-trained language model for English Tweetscited by 346
- 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
- 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
- 2019Predicting the Type and Target of Offensive Posts in Social Mediacited by 268
- 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 249
- 2020Experience Grounds Languagecited by 241
- 2019SemEval-2017 Task 4: Sentiment Analysis in Twittercited by 209
- 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 206
- 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
- 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
- 2017Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasmcited by 117
- 2021Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoderscited by 88
- 2020Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approachcited by 83
- 2020On the Sentence Embeddings from Pre-trained Language Modelscited by 83
- 2021Weakly Supervised Contrastive Learningcited by 81
- 2017On Sampling Strategies for Neural Network-based Collaborative Filteringcited by 71
- 2020SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysiscited by 54
- 2021CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understandingcited by 50
- 2021Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuningcited by 46
- 2021Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classificationcited by 43
- 2017Creating and Characterizing a Diverse Corpus of Sarcasm in Dialoguecited by 42
- 2021Improved Text Classification via Contrastive Adversarial Trainingcited by 38
- 2021Pairwise Supervised Contrastive Learning of Sentence Representationscited by 31
- 2020CrisisBERT: a Robust Transformer for Crisis Classification and Contextual Crisis Embeddingcited by 25
- 2022Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embeddingcited by 25
- 2018Twitter Sentiment Analysis via Bi-sense Emoji Embedding and Attention-based LSTMcited by 17
- 2019AraNet: A Deep Learning Toolkit for Arabic Social Mediacited by 14
- 2021Improving Social Meaning Detection with Pragmatic Masking and Surrogate Fine-Tuningcited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.