# SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics

arXiv: https://arxiv.org/abs/2005.04114 · HTML: https://mlsanity.com/paper/2005.04114/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2020.

## Built on by (most cited first, 30 of 31)

- [Compositional Human-Scene Interaction Synthesis with Semantic Control](https://mlsanity.com/paper/2207.12824/) (2022)
- [EmoLLMs: A Series of Emotional Large Language Models and Annotation Tools for Comprehensive Affective Analysis](https://mlsanity.com/paper/2401.08508/) (2024)
- [Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification](https://mlsanity.com/paper/2109.05427/) (2021)
- [A Survey of Knowledge-Intensive NLP with Pre-Trained Language Models](https://mlsanity.com/paper/2202.08772/) (2022)
- [RAEmoLLM: Retrieval Augmented LLMs for Cross-Domain Misinformation Detection Using In-Context Learning Based on Emotional Information](https://mlsanity.com/paper/2406.11093/) (2024)
- [Label-Aware Hyperbolic Embeddings for Fine-grained Emotion Classification](https://mlsanity.com/paper/2306.14822/) (2023)
- ["This is Fake! Shared it by Mistake": Assessing the Intent of Fake News Spreaders](https://mlsanity.com/paper/2202.04752/) (2022)
- [Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis](https://mlsanity.com/paper/2210.09803/) (2022)
- [Rumour Detection via Zero-shot Cross-lingual Transfer Learning](https://mlsanity.com/paper/2109.12773/) (2021)
- [Which side are you on? Insider-Outsider classification in conspiracy-theoretic social media](https://mlsanity.com/paper/2203.04356/) (2022)
- [SOUL: Towards Sentiment and Opinion Understanding of Language](https://mlsanity.com/paper/2310.17924/) (2023)
- [Representing Syntax and Composition with Geometric Transformations](https://mlsanity.com/paper/2106.01904/) (2021)
- [From Chatter to Matter: Addressing Critical Steps of Emotion Recognition Learning in Task-oriented Dialogue](https://mlsanity.com/paper/2308.12648/) (2023)
- [SentiCSE: A Sentiment-aware Contrastive Sentence Embedding Framework with Sentiment-guided Textual Similarity](https://mlsanity.com/paper/2404.01104/) (2024)
- [Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning](https://mlsanity.com/paper/2310.18930/) (2023)
- [Learning to Play Chess from Textbooks (LEAP): a Corpus for Evaluating Chess Moves based on Sentiment Analysis](https://mlsanity.com/paper/2310.20260/) (2023)
- [Non-Compositionality in Sentiment: New Data and Analyses](https://mlsanity.com/paper/2310.20656/) (2023)
- [Watch the Speakers: A Hybrid Continuous Attribution Network for Emotion Recognition in Conversation With Emotion Disentanglement](https://mlsanity.com/paper/2309.09799/) (2023)
- [Empowering Sign Language Communication: Integrating Sentiment and Semantics for Facial Expression Synthesis](https://mlsanity.com/paper/2408.15159/) (2024)
- [Evaluating and explaining training strategies for zero-shot cross-lingual news sentiment analysis](https://mlsanity.com/paper/2409.20054/) (2024)
- [1Cademy at Semeval-2022 Task 1: Investigating the Effectiveness of Multilingual, Multitask, and Language-Agnostic Tricks for the Reverse Dictionary Task](https://mlsanity.com/paper/2206.03702/) (2022)
- [The Contribution of Lyrics and Acoustics to Collaborative Understanding of Mood](https://mlsanity.com/paper/2207.05680/) (2022)
- [Faith: An Efficient Framework for Transformer Verification on GPUs](https://mlsanity.com/paper/2209.12708/) (2022)
- [Emotion Detection with Transformers: A Comparative Study](https://mlsanity.com/paper/2403.15454/) (2024)
- [SensoryT5: Infusing Sensorimotor Norms into T5 for Enhanced Fine-grained Emotion Classification](https://mlsanity.com/paper/2403.15574/) (2024)
- [SoftMCL: Soft Momentum Contrastive Learning for Fine-grained Sentiment-aware Pre-training](https://mlsanity.com/paper/2405.01827/) (2024)
- ["Yeah Right!" -- Do LLMs Exhibit Multimodal Feature Transfer?](https://mlsanity.com/paper/2501.04138/) (2025)
- [Continuous Adversarial Text Representation Learning for Affective Recognition](https://mlsanity.com/paper/2502.20613/) (2025)
- [Busemann energy-based attention for emotion analysis in Poincar'e discs](https://mlsanity.com/paper/2604.06752/) (2026)
- [Exploiting Pre-trained Encoder-Decoder Transformers for Sequence-to-Sequence Constituent Parsing](https://mlsanity.com/paper/2605.13373/) (2026)

## Abstract (arXiv)

We propose SentiBERT, a variant of BERT that effectively captures compositional sentiment semantics. The model incorporates contextualized representation with binary constituency parse tree to capture semantic composition. Comprehensive experiments demonstrate that SentiBERT achieves competitive performance on phrase-level sentiment classification. We further demonstrate that the sentiment composition learned from the phrase-level annotations on SST can be transferred to other sentiment analysis tasks as well as related tasks, such as emotion classification tasks. Moreover, we conduct ablation studies and design visualization methods to understand SentiBERT. We show that SentiBERT is better than baseline approaches in capturing negation and the contrastive relation and model the compositional sentiment semantics.