# Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding

arXiv: https://arxiv.org/abs/2412.17837 · HTML: https://mlsanity.com/paper/2412.17837/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 26 of its 76 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://mlsanity.com/paper/2402.07827/) (2024)
- [Sentiment Analysis in the Era of Large Language Models: A Reality Check](https://mlsanity.com/paper/2305.15005/) (2023)
- [Crowdsourcing a Word-Emotion Association Lexicon](https://mlsanity.com/paper/1308.6297/) (2013)
- [Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT](https://mlsanity.com/paper/2302.10198/) (2023)
- [SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects](https://mlsanity.com/paper/2309.07445/) (2023)
- [Do Multilingual Language Models Think Better in English?](https://mlsanity.com/paper/2308.01223/) (2023)
- [AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages](https://mlsanity.com/paper/2302.08956/) (2023)
- [POTATO: The Portable Text Annotation Tool](https://mlsanity.com/paper/2212.08620/) (2022)
- [EmoLLMs: A Series of Emotional Large Language Models and Annotation Tools for Comprehensive Affective Analysis](https://mlsanity.com/paper/2401.08508/) (2024)
- [AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languages](https://mlsanity.com/paper/2211.03263/) (2022)
- [What is the Role of Small Models in the LLM Era: A Survey](https://mlsanity.com/paper/2409.06857/) (2024)
- [Language Models for Text Classification: Is In-Context Learning Enough?](https://mlsanity.com/paper/2403.17661/) (2024)
- [The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional Analysis](https://mlsanity.com/paper/2402.12976/) (2024)
- [Introducing various Semantic Models for Amharic: Experimentation and Evaluation with multiple Tasks and Datasets](https://mlsanity.com/paper/2011.01154/) (2020)
- [SemEval-2024 Task 3: Multimodal Emotion Cause Analysis in Conversations](https://mlsanity.com/paper/2405.13049/) (2024)
- [Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya](https://mlsanity.com/paper/2006.07698/) (2020)
- [EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluation](https://mlsanity.com/paper/2403.13737/) (2024)
- [EmoInHindi: A Multi-label Emotion and Intensity Annotated Dataset in Hindi for Emotion Recognition in Dialogues](https://mlsanity.com/paper/2205.13908/) (2022)

## Built on by (most cited first, 17 of 17)

- [AfroXLMR-Social: Adapting Pre-trained Language Models for African Languages Social Media Text](https://mlsanity.com/paper/2503.18247/) (2025)
- [AfriMTEB and AfriE5: Benchmarking and Adapting Text Embedding Models for African Languages](https://mlsanity.com/paper/2510.23896/) (2025)
- [Team A at SemEval-2025 Task 11: Breaking Language Barriers in Emotion Detection with Multilingual Models](https://mlsanity.com/paper/2502.19856/) (2025)
- [Lotus at SemEval-2025 Task 11: RoBERTa with Llama-3 Generated Explanations for Multi-Label Emotion Classification](https://mlsanity.com/paper/2502.19935/) (2025)
- [Enhancing Multi-Label Emotion Analysis and Corresponding Intensities for Ethiopian Languages](https://mlsanity.com/paper/2503.18253/) (2025)
- [Exploring Cultural Nuances in Emotion Perception Across 15 African Languages](https://mlsanity.com/paper/2503.19642/) (2025)
- [UoB-NLP at SemEval-2025 Task 11: Leveraging Adapters for Multilingual and Cross-Lingual Emotion Detection](https://mlsanity.com/paper/2504.08543/) (2025)
- [JNLP at SemEval-2025 Task 11: Cross-Lingual Multi-Label Emotion Detection Using Generative Models](https://mlsanity.com/paper/2505.13244/) (2025)
- [University of Indonesia at SemEval-2025 Task 11: Evaluating State-of-the-Art Encoders for Multi-Label Emotion Detection](https://mlsanity.com/paper/2505.16460/) (2025)
- [Empaths at SemEval-2025 Task 11: Retrieval-Augmented Approach to Perceived Emotions Prediction](https://mlsanity.com/paper/2506.04409/) (2025)
- [PromotionGo at SemEval-2025 Task 11: A Feature-Centric Framework for Cross-Lingual Multi-Emotion Detection in Short Texts](https://mlsanity.com/paper/2507.08499/) (2025)
- [Chinchunmei at SemEval-2025 Task 11: Boosting the Large Language Model's Capability of Emotion Perception using Contrastive Learning](https://mlsanity.com/paper/2507.15714/) (2025)
- [CSIRO-LT at SemEval-2025 Task 11: Adapting LLMs for Emotion Recognition for Multiple Languages](https://mlsanity.com/paper/2508.01161/) (2025)
- [Emotion Entanglement and Bayesian Inference for Multi-Dimensional Emotion Understanding](https://mlsanity.com/paper/2604.00819/) (2026)
- [Temporal Simultaneity Predicts Annotation Quality in Sentiment Corpora](https://mlsanity.com/paper/2605.27239/) (2026)
- [Cross-lingual Functional Vectors for Emotion Detection in Large Language Models](https://mlsanity.com/paper/2608.29613/) (2026)
- [YNU-HPCC at SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Using Multiple Prediction Headers](https://mlsanity.com/paper/2609.19238/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) show promising learning and reasoning abilities. Compared to other NLP tasks, multilingual and multi-label emotion evaluation tasks are under-explored in LLMs. In this paper, we present EthioEmo, a multi-label emotion classification dataset for four Ethiopian languages, namely, Amharic (amh), Afan Oromo (orm), Somali (som), and Tigrinya (tir). We perform extensive experiments with an additional English multi-label emotion dataset from SemEval 2018 Task 1. Our evaluation includes encoder-only, encoder-decoder, and decoder-only language models. We compare zero and few-shot approaches of LLMs to fine-tuning smaller language models. The results show that accurate multi-label emotion classification is still insufficient even for high-resource languages such as English, and there is a large gap between the performance of high-resource and low-resource languages. The results also show varying performance levels depending on the language and model type. EthioEmo is available publicly to further improve the understanding of emotions in language models and how people convey emotions through various languages.