# Improving Arabic Multi-Label Emotion Classification using Stacked Embeddings and Hybrid Loss Function

arXiv: https://arxiv.org/abs/2410.03979 · HTML: https://mlsanity.com/paper/2410.03979/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 74 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [FastText.zip: Compressing text classification models](https://mlsanity.com/paper/1612.03651/) (2016)
- [Dice Loss for Data-imbalanced NLP Tasks](https://mlsanity.com/paper/1911.02855/) (2019)
- [ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic](https://mlsanity.com/paper/2101.01785/) (2020)
- [Learning a Deep ConvNet for Multi-label Classification with Partial Labels](https://mlsanity.com/paper/1902.09720/) (2019)
- [KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media](https://mlsanity.com/paper/2007.13184/) (2020)
- [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://mlsanity.com/paper/2102.10684/) (2021)
- [SpanEmo: Casting Multi-label Emotion Classification as Span-prediction](https://mlsanity.com/paper/2101.10038/) (2021)
- [An Ensemble Method to Produce High-Quality Word Embeddings (2016)](https://mlsanity.com/paper/1604.01692/) (2016)
- [EmoGraph: Capturing Emotion Correlations using Graph Networks](https://mlsanity.com/paper/2008.09378/) (2020)
- [An Effective Deployment of Contrastive Learning in Multi-label Text Classification](https://mlsanity.com/paper/2212.00552/) (2022)
- [A Multi-level Supervised Contrastive Learning Framework for Low-Resource Natural Language Inference](https://mlsanity.com/paper/2205.15550/) (2022)
- [Linear Ensembles of Word Embedding Models](https://mlsanity.com/paper/1704.01419/) (2017)

## Built on by (most cited first, 1 of 1)

- [A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots](https://mlsanity.com/paper/2606.19660/) (2026)

## Abstract (arXiv)

In multi-label emotion classification, particularly for low-resource languages like Arabic, the challenges of class imbalance and label correlation hinder model performance, especially in accurately predicting minority emotions. To address these issues, this study proposes a novel approach that combines stacked embeddings, meta-learning, and a hybrid loss function to enhance multi-label emotion classification for the Arabic language. The study extracts contextual embeddings from three fine-tuned language models-ArabicBERT, MarBERT, and AraBERT-which are then stacked to form enriched embeddings. A meta-learner is trained on these stacked embeddings, and the resulting concatenated representations are provided as input to a Bi-LSTM model, followed by a fully connected neural network for multi-label classification. To further improve performance, a hybrid loss function is introduced, incorporating class weighting, label correlation matrix, and contrastive learning, effectively addressing class imbalances and improving the handling of label correlations. Extensive experiments validate the proposed model's performance across key metrics such as Precision, Recall, F1-Score, Jaccard Accuracy, and Hamming Loss. The class-wise performance analysis demonstrates the hybrid loss function's ability to significantly reduce disparities between majority and minority classes, resulting in a more balanced emotion classification. An ablation study highlights the contribution of each component, showing the superiority of the model compared to baseline approaches and other loss functions. This study not only advances multi-label emotion classification for Arabic but also presents a generalizable framework that can be adapted to other languages and domains, providing a significant step forward in addressing the challenges of low-resource emotion classification tasks.