# EmoBench-UA: A Benchmark Dataset for Emotion Detection in Ukrainian

arXiv: https://arxiv.org/abs/2505.23297 · HTML: https://mlsanity.com/paper/2505.23297/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 29 May 2025.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [AdapterHub: A Framework for Adapting Transformers](https://mlsanity.com/paper/2007.07779/) (2020)
- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://mlsanity.com/paper/2402.07827/) (2024)
- [Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages](https://mlsanity.com/paper/2305.12182/) (2023)
- [XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond](https://mlsanity.com/paper/2104.12250/) (2021)
- [EuroLLM: Multilingual Language Models for Europe](https://mlsanity.com/paper/2409.16235/) (2024)
- [XED: A Multilingual Dataset for Sentiment Analysis and Emotion Detection](https://mlsanity.com/paper/2011.01612/) (2020)
- [Load What You Need: Smaller Versions of Multilingual BERT](https://mlsanity.com/paper/2010.05609/) (2020)
- [Best Practices in the Creation and Use of Emotion Lexicons](https://mlsanity.com/paper/2210.07206/) (2022)
- [BRIGHTER: BRIdging the Gap in Human-Annotated Textual Emotion Recognition Datasets for 28 Languages](https://mlsanity.com/paper/2502.11926/) (2025)
- [A Survey on Aspect-Based Sentiment Classification](https://mlsanity.com/paper/2203.14266/) (2022)
- [Discovering Emotion and Reasoning its Flip in Multi-Party Conversations using Masked Memory Network and Transformer](https://mlsanity.com/paper/2103.12360/) (2021)
- [Detecting Text Formality: A Study of Text Classification Approaches](https://mlsanity.com/paper/2204.08975/) (2022)
- [SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection](https://mlsanity.com/paper/2503.07269/) (2025)
- [Toxicity Classification in Ukrainian](https://mlsanity.com/paper/2404.17841/) (2024)
- [Cross-lingual Text Classification Transfer: The Case of Ukrainian](https://mlsanity.com/paper/2404.02043/) (2024)
- [Spivavtor: An Instruction Tuned Ukrainian Text Editing Model](https://mlsanity.com/paper/2404.18880/) (2024)

## Abstract (arXiv)

While Ukrainian NLP has seen progress in many texts processing tasks, emotion classification remains an underexplored area with no publicly available benchmark to date. In this work, we introduce EmoBench-UA, the first annotated dataset for emotion detection in Ukrainian texts. Our annotation schema is adapted from the previous English-centric works on emotion detection (Mohammad et al., 2018; Mohammad, 2022) guidelines. The dataset was created through crowdsourcing using the Toloka.ai platform ensuring high-quality of the annotation process. Then, we evaluate a range of approaches on the collected dataset, starting from linguistic-based baselines, synthetic data translated from English, to large language models (LLMs). Our findings highlight the challenges of emotion classification in non-mainstream languages like Ukrainian and emphasize the need for further development of Ukrainian-specific models and training resources.