mlsanityData to 7 Oct 2026

NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis

arXiv 2305.00090 ↗First posted 28 Apr 2023cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

This paper describes our system developed for the SemEval-2023 Task 12 "Sentiment Analysis for Low-resource African Languages using Twitter Dataset". Sentiment analysis is one of the most widely studied applications in natural language processing. However, most prior work still focuses on a small number of high-resource languages. Building reliable sentiment analysis systems for low-resource languages remains challenging, due to the limited training data in this task. In this work, we propose to leverage language-adaptive and task-adaptive pretraining on African texts and study transfer learning with source language selection on top of an African language-centric pretrained language model. Our key findings are: (1) Adapting the pretrained model to the target language and task using a small yet relevant corpus improves performance remarkably by more than 10 F1 score points. (2) Selecting source languages with positive transfer gains during training can avoid harmful interference from dissimilar languages, leading to better results in multilingual and cross-lingual settings. In the shared task, our system wins 8 out of 15 tracks and, in particular, performs best in the multilingual evaluation.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 28 Apr 2023.

Builds on

16 of 25 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  4. 2020BERTweet: A pre-trained language model for English Tweetscited by 346
  5. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
  6. 2022Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuningcited by 149
  7. 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 143
  8. 2023AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languagescited by 68
  9. 2017Learning to select data for transfer learning with Bayesian Optimizationcited by 64
  10. 2022AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languagescited by 46
  11. 2023SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)cited by 41
  12. 2022MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognitioncited by 38
4 more
  1. 2017Forward-Backward Selection with Early Droppingcited by 18
  2. 2021To Share or not to Share: Predicting Sets of Sources for Model Transfer Learningcited by 11
  3. 2021CLIN-X: pre-trained language models and a study on cross-task transfer for concept extraction in the clinical domaincited by 7
  4. 2022Geographical Distance Is The New Hyperparameter: A Case Study Of Finding The Optimal Pre-trained Language For English-isiZulu Machine Translationcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 4; 2025: 1.

10 papers.

  1. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  2. 2024Goldfish: Monolingual Language Models for 350 Languagescited by 27
  3. 2024Rehearsal-Free Modular and Compositional Continual Learning for Language Modelscited by 26
  4. 2024EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluationcited by 8
  5. 2023TADA: Efficient Task-Agnostic Domain Adaptation for Transformerscited by 6
  6. 2023GradSim: Gradient-Based Language Grouping for Effective Multilingual Trainingcited by 6
  7. 2024XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examplescited by 5
  8. 2025AfroXLMR-Social: Adapting Pre-trained Language Models for African Languages Social Media Textcited by 4
  9. 2023LowResource at BLP-2023 Task 2: Leveraging BanglaBert for Low Resource Sentiment Analysis of Bangla Languagecited by 3
  10. 2023mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Modelscited by 0