# UCAS-IIE-NLP at SemEval-2023 Task 12: Enhancing Generalization of Multilingual BERT for Low-resource Sentiment Analysis

arXiv: https://arxiv.org/abs/2306.01093 · HTML: https://mlsanity.com/paper/2306.01093/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Adversarial Training Methods for Semi-Supervised Text Classification](https://mlsanity.com/paper/1605.07725/) (2016)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning](https://mlsanity.com/paper/2204.06487/) (2022)
- [DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversations](https://mlsanity.com/paper/2106.01978/) (2021)
- [AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages](https://mlsanity.com/paper/2302.08956/) (2023)
- [MM-DFN: Multimodal Dynamic Fusion Network for Emotion Recognition in Conversations](https://mlsanity.com/paper/2203.02385/) (2022)
- [Towards Afrocentric NLP for African Languages: Where We Are and Where We Can Go](https://mlsanity.com/paper/2203.08351/) (2022)
- [SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)](https://mlsanity.com/paper/2304.06845/) (2023)
- [VarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understanding](https://mlsanity.com/paper/2211.00430/) (2022)
- [A review of sentiment analysis research in Arabic language](https://mlsanity.com/paper/2005.12240/) (2020)
- [Hierarchical Interaction Networks with Rethinking Mechanism for Document-level Sentiment Analysis](https://mlsanity.com/paper/2007.08445/) (2020)
- [PEN4Rec: Preference Evolution Networks for Session-based Recommendation](https://mlsanity.com/paper/2106.09306/) (2021)

## Abstract (arXiv)

This paper describes our system designed for SemEval-2023 Task 12: Sentiment analysis for African languages. The challenge faced by this task is the scarcity of labeled data and linguistic resources in low-resource settings. To alleviate these, we propose a generalized multilingual system SACL-XLMR for sentiment analysis on low-resource languages. Specifically, we design a lexicon-based multilingual BERT to facilitate language adaptation and sentiment-aware representation learning. Besides, we apply a supervised adversarial contrastive learning technique to learn sentiment-spread structured representations and enhance model generalization. Our system achieved competitive results, largely outperforming baselines on both multilingual and zero-shot sentiment classification subtasks. Notably, the system obtained the 1st rank on the zero-shot classification subtask in the official ranking. Extensive experiments demonstrate the effectiveness of our system.