# REDAffectiveLM: Leveraging Affect Enriched Embedding and Transformer-based Neural Language Model for Readers' Emotion Detection

arXiv: https://arxiv.org/abs/2301.08995 · HTML: https://mlsanity.com/paper/2301.08995/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 68 references that are in the map.
- First posted on arXiv 21 Jan 2023.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Counter-fitting Word Vectors to Linguistic Constraints](https://mlsanity.com/paper/1603.00892/) (2016)
- [DialogXL: All-in-One XLNet for Multi-Party Conversation Emotion Recognition](https://mlsanity.com/paper/2012.08695/) (2020)
- [Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detection](https://mlsanity.com/paper/2003.10769/) (2020)
- [PO-EMO: Conceptualization, Annotation, and Modeling of Aesthetic Emotions in German and English Poetry](https://mlsanity.com/paper/2003.07723/) (2020)
- [Deep learning for affective computing: text-based emotion recognition in decision support](https://mlsanity.com/paper/1803.06397/) (2018)
- [Learning and Evaluating Emotion Lexicons for 91 Languages](https://mlsanity.com/paper/2005.05672/) (2020)
- [DepecheMood++: a Bilingual Emotion Lexicon Built Through Simple Yet Powerful Techniques](https://mlsanity.com/paper/1810.03660/) (2018)
- [Emotion Cognizance Improves Health Fake News Identification](https://mlsanity.com/paper/1906.10365/) (2019)
- [BERT-ERC: Fine-tuning BERT is Enough for Emotion Recognition in Conversation](https://mlsanity.com/paper/2301.06745/) (2023)
- [Towards an Enhanced Understanding of Bias in Pre-trained Neural Language Models: A Survey with Special Emphasis on Affective Bias](https://mlsanity.com/paper/2204.10365/) (2022)
- [Emotional Embeddings: Refining Word Embeddings to Capture Emotional Content of Words](https://mlsanity.com/paper/1906.00112/) (2019)
- [Social Emotion Mining Techniques for Facebook Posts Reaction Prediction](https://mlsanity.com/paper/1712.03249/) (2017)
- [Deep Feelings: A Massive Cross-Lingual Study on the Relation between Emotions and Virality](https://mlsanity.com/paper/1503.04723/) (2015)
- [Aff2Vec: Affect--Enriched Distributional Word Representations](https://mlsanity.com/paper/1805.07966/) (2018)
- [Language Models as Emotional Classifiers for Textual Conversations](https://mlsanity.com/paper/2008.12360/) (2020)
- [AC-BLSTM: Asymmetric Convolutional Bidirectional LSTM Networks for Text Classification](https://mlsanity.com/paper/1611.01884/) (2016)

## Abstract (arXiv)

Technological advancements in web platforms allow people to express and share emotions towards textual write-ups written and shared by others. This brings about different interesting domains for analysis; emotion expressed by the writer and emotion elicited from the readers. In this paper, we propose a novel approach for Readers' Emotion Detection from short-text documents using a deep learning model called REDAffectiveLM. Within state-of-the-art NLP tasks, it is well understood that utilizing context-specific representations from transformer-based pre-trained language models helps achieve improved performance. Within this affective computing task, we explore how incorporating affective information can further enhance performance. Towards this, we leverage context-specific and affect enriched representations by using a transformer-based pre-trained language model in tandem with affect enriched Bi-LSTM+Attention. For empirical evaluation, we procure a new dataset REN-20k, besides using RENh-4k and SemEval-2007. We evaluate the performance of our REDAffectiveLM rigorously across these datasets, against a vast set of state-of-the-art baselines, where our model consistently outperforms baselines and obtains statistically significant results. Our results establish that utilizing affect enriched representation along with context-specific representation within a neural architecture can considerably enhance readers' emotion detection. Since the impact of affect enrichment specifically in readers' emotion detection isn't well explored, we conduct a detailed analysis over affect enriched Bi-LSTM+Attention using qualitative and quantitative model behavior evaluation techniques. We observe that compared to conventional semantic embedding, affect enriched embedding increases ability of the network to effectively identify and assign weightage to key terms responsible for readers' emotion detection.