# Emotions are Subtle: Learning Sentiment Based Text Representations Using Contrastive Learning

arXiv: https://arxiv.org/abs/2112.01054 · HTML: https://mlsanity.com/paper/2112.01054/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Whitening Sentence Representations for Better Semantics and Faster Retrieval](https://mlsanity.com/paper/2103.15316/) (2021)
- [ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding](https://mlsanity.com/paper/2109.04380/) (2021)
- [DynaSent: A Dynamic Benchmark for Sentiment Analysis](https://mlsanity.com/paper/2012.15349/) (2020)
- [Contrastive Learning for Fair Representations](https://mlsanity.com/paper/2109.10645/) (2021)
- [The Evolution of Sentiment Analysis - A Review of Research Topics, Venues, and Top Cited Papers](https://mlsanity.com/paper/1612.01556/) (2016)
- [Adversarial Training with Contrastive Learning in NLP](https://mlsanity.com/paper/2109.09075/) (2021)
- [Sentence Embeddings using Supervised Contrastive Learning](https://mlsanity.com/paper/2106.04791/) (2021)

## Built on by (most cited first, 1 of 1)

- [DEFTri: A Few-Shot Label Fused Contextual Representation Learning For Product Defect Triage in e-Commerce](https://mlsanity.com/paper/2307.11344/) (2023)

## Abstract (arXiv)

Contrastive learning techniques have been widely used in the field of computer vision as a means of augmenting datasets. In this paper, we extend the use of these contrastive learning embeddings to sentiment analysis tasks and demonstrate that fine-tuning on these embeddings provides an improvement over fine-tuning on BERT-based embeddings to achieve higher benchmarks on the task of sentiment analysis when evaluated on the DynaSent dataset. We also explore how our fine-tuned models perform on cross-domain benchmark datasets. Additionally, we explore upsampling techniques to achieve a more balanced class distribution to make further improvements on our benchmark tasks.