# Dimensional Modeling of Emotions in Text with Appraisal Theories: Corpus Creation, Annotation Reliability, and Prediction

arXiv: https://arxiv.org/abs/2206.05238 · HTML: https://mlsanity.com/paper/2206.05238/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 134 references that are in the map.
- First posted on arXiv 10 Jun 2022.

## Builds on (most cited first, 23 of 23)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Crowdsourcing a Word-Emotion Association Lexicon](https://mlsanity.com/paper/1308.6297/) (2013)
- [Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm](https://mlsanity.com/paper/1708.00524/) (2017)
- [Emotion-Cause Pair Extraction: A New Task to Emotion Analysis in Texts](https://mlsanity.com/paper/1906.01267/) (2019)
- [EmoBank: Studying the Impact of Annotation Perspective and Representation Format on Dimensional Emotion Analysis](https://mlsanity.com/paper/2205.01996/) (2022)
- [GoodNewsEveryone: A Corpus of News Headlines Annotated with Emotions, Semantic Roles, and Reader Perception](https://mlsanity.com/paper/1912.03184/) (2019)
- [Appraisal Theories for Emotion Classification in Text](https://mlsanity.com/paper/2003.14155/) (2020)
- [Crowdsourcing and Validating Event-focused Emotion Corpora for German and English](https://mlsanity.com/paper/1905.13618/) (2019)
- [Dimensional Emotion Detection from Categorical Emotion](https://mlsanity.com/paper/1911.02499/) (2019)
- [Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysis](https://mlsanity.com/paper/2109.08256/) (2021)
- [PO-EMO: Conceptualization, Annotation, and Modeling of Aesthetic Emotions in German and English Poetry](https://mlsanity.com/paper/2003.07723/) (2020)
- [Learning and Evaluating Emotion Lexicons for 91 Languages](https://mlsanity.com/paper/2005.05672/) (2020)
- [Emotion Recognition under Consideration of the Emotion Component Process Model](https://mlsanity.com/paper/2107.12895/) (2021)
- [IEST: WASSA-2018 Implicit Emotions Shared Task](https://mlsanity.com/paper/1809.01083/) (2018)
- [Towards Label-Agnostic Emotion Embeddings](https://mlsanity.com/paper/2012.00190/) (2020)
- [x-enVENT: A Corpus of Event Descriptions with Experiencer-specific Emotion and Appraisal Annotations](https://mlsanity.com/paper/2203.10909/) (2022)
- [APPReddit: a Corpus of Reddit Posts Annotated for Appraisal](https://mlsanity.com/paper/2205.15627/) (2022)
- [Emotion-Aware, Emotion-Agnostic, or Automatic: Corpus Creation Strategies to Obtain Cognitive Event Appraisal Annotations](https://mlsanity.com/paper/2102.12858/) (2021)
- [Understanding the Role of Affect Dimensions in Detecting Emotions from Tweets: A Multi-task Approach](https://mlsanity.com/paper/2105.03983/) (2021)

## Built on by (most cited first, 30 of 46)

- [Emotion Analysis in NLP: Trends, Gaps and Roadmap for Future Directions](https://mlsanity.com/paper/2403.01222/) (2024)
- [Mechanistic Interpretability of Emotion Inference in Large Language Models](https://mlsanity.com/paper/2502.05489/) (2025)
- [GPT-4 Emulates Average-Human Emotional Cognition from a Third-Person Perspective](https://mlsanity.com/paper/2408.13718/) (2024)
- [Natural Language Inference Prompts for Zero-shot Emotion Classification in Text across Corpora](https://mlsanity.com/paper/2209.06701/) (2022)
- [Fearful Falcons and Angry Llamas: Emotion Category Annotations of Arguments by Humans and LLMs](https://mlsanity.com/paper/2412.15993/) (2024)
- [Which Demographics do LLMs Default to During Annotation?](https://mlsanity.com/paper/2410.08820/) (2024)
- [Emotion Recognition based on Psychological Components in Guided Narratives for Emotion Regulation](https://mlsanity.com/paper/2305.10446/) (2023)
- [Where are We in Event-centric Emotion Analysis? Bridging Emotion Role Labeling and Appraisal-based Approaches](https://mlsanity.com/paper/2309.02092/) (2023)
- [Toward Artificial Empathy for Human-Centered Design: A Framework](https://mlsanity.com/paper/2303.10583/) (2023)
- [Emotion-Conditioned Text Generation through Automatic Prompt Optimization](https://mlsanity.com/paper/2308.04857/) (2023)
- [Language Models Predict Empathy Gaps Between Social In-groups and Out-groups](https://mlsanity.com/paper/2503.01030/) (2025)
- [Experiencer-Specific Emotion and Appraisal Prediction](https://mlsanity.com/paper/2210.12078/) (2022)
- [Affective Natural Language Generation of Event Descriptions through Fine-grained Appraisal Conditions](https://mlsanity.com/paper/2307.14004/) (2023)
- [Improving Language Models for Emotion Analysis: Insights from Cognitive Science](https://mlsanity.com/paper/2406.10265/) (2024)
- [Zero-shot Emotion Annotation in Facial Images Using Large Multimodal Models: Benchmarking and Prospects for Multi-Class, Multi-Frame Approaches](https://mlsanity.com/paper/2502.12454/) (2025)
- [Modeling Subjectivity in Cognitive Appraisal with Language Models](https://mlsanity.com/paper/2503.11381/) (2025)
- [Exploratory Study into Relations between Cognitive Distortions and Emotional Appraisals](https://mlsanity.com/paper/2503.15979/) (2025)
- [LLM-based Affective Text Generation Quality Based on Different Quantization Values](https://mlsanity.com/paper/2501.19317/) (2025)
- [Investigating Subjective Factors of Argument Strength: Storytelling, Emotions, and Hedging](https://mlsanity.com/paper/2507.17409/) (2025)
- [Trust Me, I Can Convince You: The Contextualized Argument Appraisal Framework](https://mlsanity.com/paper/2509.17844/) (2025)
- [CAREBench: Evaluating LLMs' Emotion Understanding by Assessing Cognitive Appraisal Reasoning](https://mlsanity.com/paper/2605.17176/) (2026)
- [Automatic Emotion Experiencer Recognition](https://mlsanity.com/paper/2305.16731/) (2023)
- [Why Robots Are Bad at Detecting Their Mistakes: Limitations of Miscommunication Detection in Human-Robot Dialogue](https://mlsanity.com/paper/2506.20268/) (2025)
- [Whether, Not Which: Mechanistic Interpretability Reveals Dissociable Affect Reception and Emotion Categorization in LLMs](https://mlsanity.com/paper/2603.22295/) (2026)
- [Why Are Some Emotions Harder for LLMs? Uncovering the Causal Mechanisms of Emotion Inference via Sparse Autoencoders](https://mlsanity.com/paper/2604.25866/) (2026)
- [Topic Bias in Emotion Classification](https://mlsanity.com/paper/2312.09043/) (2023)
- [Emotion Identification for French in Written Texts: Considering their Modes of Expression as a Step Towards Text Complexity Analysis](https://mlsanity.com/paper/2405.14385/) (2024)
- [Assessing the Reliability and Validity of GPT-4 in Annotating Emotion Appraisal Ratings](https://mlsanity.com/paper/2503.16883/) (2025)
- [Donate or Create? Comparing Data Collection Strategies for Emotion-labeled Multimodal Social Media Posts](https://mlsanity.com/paper/2505.24427/) (2025)
- [Salience Adjustment for Context-Based Emotion Recognition](https://mlsanity.com/paper/2507.15878/) (2025)

## Abstract (arXiv)

The most prominent tasks in emotion analysis are to assign emotions to texts and to understand how emotions manifest in language. An observation for NLP is that emotions can be communicated implicitly by referring to events, appealing to an empathetic, intersubjective understanding of events, even without explicitly mentioning an emotion name. In psychology, the class of emotion theories known as appraisal theories aims at explaining the link between events and emotions. Appraisals can be formalized as variables that measure a cognitive evaluation by people living through an event that they consider relevant. They include the assessment if an event is novel, if the person considers themselves to be responsible, if it is in line with the own goals, and many others. Such appraisals explain which emotions are developed based on an event, e.g., that a novel situation can induce surprise or one with uncertain consequences could evoke fear. We analyze the suitability of appraisal theories for emotion analysis in text with the goal of understanding if appraisal concepts can reliably be reconstructed by annotators, if they can be predicted by text classifiers, and if appraisal concepts help to identify emotion categories. To achieve that, we compile a corpus by asking people to textually describe events that triggered particular emotions and to disclose their appraisals. Then, we ask readers to reconstruct emotions and appraisals from the text. This setup allows us to measure if emotions and appraisals can be recovered purely from text and provides a human baseline. Our comparison of text classification methods to human annotators shows that both can reliably detect emotions and appraisals with similar performance. Therefore, appraisals constitute an alternative computational emotion analysis paradigm and further improve the categorization of emotions in text with joint models.