# SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge

arXiv: https://arxiv.org/abs/1911.02493 · HTML: https://mlsanity.com/paper/1911.02493/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2019.

## Built on by (most cited first, 1 of 1)

- [Humor@IITK at SemEval-2021 Task 7: Large Language Models for Quantifying Humor and Offensiveness](https://mlsanity.com/paper/2104.00933/) (2021)

## Abstract (arXiv)

Most of the existing pre-trained language representation models neglect to consider the linguistic knowledge of texts, which can promote language understanding in NLP tasks. To benefit the downstream tasks in sentiment analysis, we propose a novel language representation model called SentiLARE, which introduces word-level linguistic knowledge including part-of-speech tag and sentiment polarity (inferred from SentiWordNet) into pre-trained models. We first propose a context-aware sentiment attention mechanism to acquire the sentiment polarity of each word with its part-of-speech tag by querying SentiWordNet. Then, we devise a new pre-training task called label-aware masked language model to construct knowledge-aware language representation. Experiments show that SentiLARE obtains new state-of-the-art performance on a variety of sentiment analysis tasks.