# Sentiment Analysis with Contextual Embeddings and Self-Attention

arXiv: https://arxiv.org/abs/2003.05574 · HTML: https://mlsanity.com/paper/2003.05574/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2020.

## Built on by (most cited first, 2 of 2)

- [Re-Evaluating GermEval17 Using German Pre-Trained Language Models](https://mlsanity.com/paper/2102.12330/) (2021)
- [Domain Adaptation of Transformer-Based Models using Unlabeled Data for Relevance and Polarity Classification of German Customer Feedback](https://mlsanity.com/paper/2212.05764/) (2022)

## Abstract (arXiv)

In natural language the intended meaning of a word or phrase is often implicit and depends on the context. In this work, we propose a simple yet effective method for sentiment analysis using contextual embeddings and a self-attention mechanism. The experimental results for three languages, including morphologically rich Polish and German, show that our model is comparable to or even outperforms state-of-the-art models. In all cases the superiority of models leveraging contextual embeddings is demonstrated. Finally, this work is intended as a step towards introducing a universal, multilingual sentiment classifier.