# Emo Pillars: Knowledge Distillation to Support Fine-Grained Context-Aware and Context-Less Emotion Classification

arXiv: https://arxiv.org/abs/2504.16856 · HTML: https://mlsanity.com/paper/2504.16856/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal distillation, one of 12 directions in Knowledge distillation.
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 23 Apr 2025.

## Placed in

- Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/)

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [ChatGPT: Jack of all trades, master of none](https://mlsanity.com/paper/2302.10724/) (2023)
- [Dimensional Emotion Detection from Categorical Emotion](https://mlsanity.com/paper/1911.02499/) (2019)
- [Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interaction](https://mlsanity.com/paper/2311.04507/) (2023)
- [ANA at SemEval-2019 Task 3: Contextual Emotion detection in Conversations through hierarchical LSTMs and BERT](https://mlsanity.com/paper/1904.00132/) (2019)
- [Towards Coherent and Consistent Use of Entities in Narrative Generation](https://mlsanity.com/paper/2202.01709/) (2022)
- [The Strong Pull of Prior Knowledge in Large Language Models and Its Impact on Emotion Recognition](https://mlsanity.com/paper/2403.17125/) (2024)
- [Uncovering the Limits of Text-based Emotion Detection](https://mlsanity.com/paper/2109.01900/) (2021)
- [Sharing emotions at scale: The Vent dataset](https://mlsanity.com/paper/1901.04856/) (2019)
- [From Big to Small Without Losing It All: Text Augmentation with ChatGPT for Efficient Sentiment Analysis](https://mlsanity.com/paper/2312.04720/) (2023)
- [Data Augmentation for Emotion Detection in Small Imbalanced Text Data](https://mlsanity.com/paper/2310.17015/) (2023)

## Built on by (most cited first, 4 of 4)

- [Contrastive Distillation of Emotion Knowledge from LLMs for Zero-Shot Emotion Recognition](https://mlsanity.com/paper/2505.18040/) (2025)
- [Language Models Might Not Understand You: Evaluating Theory of Mind via Story Prompting](https://mlsanity.com/paper/2506.19089/) (2025)
- [Chiaroscuro for Emotions: A Contrastive Emotion Benchmark Grounded in Appraisal Theory](https://mlsanity.com/paper/2609.03394/) (2026)
- [Emo-Jev: Probabilistic Reasoning for Emotion Classification with Jev](https://mlsanity.com/paper/2610.08829/) (2026)

## Abstract (arXiv)

Most datasets for sentiment analysis lack context in which an opinion was expressed, often crucial for emotion understanding, and are mainly limited by a few emotion categories. Foundation large language models (LLMs) like GPT-4 suffer from over-predicting emotions and are too resource-intensive. We design an LLM-based data synthesis pipeline and leverage a large model, Mistral-7b, for the generation of training examples for more accessible, lightweight BERT-type encoder models. We focus on enlarging the semantic diversity of examples and propose grounding the generation into a corpus of narratives to produce non-repetitive story-character-centered utterances with unique contexts over 28 emotion classes. By running 700K inferences in 450 GPU hours, we contribute with the dataset of 100K contextual and also 300K context-less examples to cover both scenarios. We use it for fine-tuning pre-trained encoders, which results in several Emo Pillars models. We show that Emo Pillars models are highly adaptive to new domains when tuned to specific tasks such as GoEmotions, ISEAR, IEMOCAP, and EmoContext, reaching the SOTA performance on the first three. We also validate our dataset, conducting statistical analysis and human evaluation, and confirm the success of our measures in utterance diversification (although less for the neutral class) and context personalization, while pointing out the need for improved handling of out-of-taxonomy labels within the pipeline.