# Solution for Emotion Prediction Competition of Workshop on Emotionally and Culturally Intelligent AI

arXiv: https://arxiv.org/abs/2403.17683 · HTML: https://mlsanity.com/paper/2403.17683/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 16 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Semi-Supervised Multi-Modal Multi-Instance Multi-Label Deep Network with Optimal Transport](https://mlsanity.com/paper/2104.08489/) (2021)
- [Exploiting Cross-Modal Prediction and Relation Consistency for Semi-Supervised Image Captioning](https://mlsanity.com/paper/2110.11767/) (2021)
- [ArtELingo: A Million Emotion Annotations of WikiArt with Emphasis on Diversity over Language and Culture](https://mlsanity.com/paper/2211.10780/) (2022)
- [X2-VLM: All-In-One Pre-trained Model For Vision-Language Tasks](https://mlsanity.com/paper/2211.12402/) (2022)
- [Describing emotions with acoustic property prompts for speech emotion recognition](https://mlsanity.com/paper/2211.07737/) (2022)

## Abstract (arXiv)

This report provide a detailed description of the method that we explored and proposed in the WECIA Emotion Prediction Competition (EPC), which predicts a person's emotion through an artistic work with a comment. The dataset of this competition is ArtELingo, designed to encourage work on diversity across languages and cultures. The dataset has two main challenges, namely modal imbalance problem and language-cultural differences problem. In order to address this issue, we propose a simple yet effective approach called single-multi modal with Emotion-Cultural specific prompt(ECSP), which focuses on using the single modal message to enhance the performance of multimodal models and a well-designed prompt to reduce cultural differences problem. To clarify, our approach contains two main blocks: (1)XLM-Rciteconneau2019unsupervised based unimodal model and X2-VLMcitezeng2022x based multimodal model (2) Emotion-Cultural specific prompt. Our approach ranked first in the final test with a score of 0.627.