# EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluation

arXiv: https://arxiv.org/abs/2403.13737 · HTML: https://mlsanity.com/paper/2403.13737/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 77 references that are in the map.
- First posted on arXiv 20 Mar 2024.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training](https://mlsanity.com/paper/2007.07834/) (2020)
- [MasakhaNER: Named Entity Recognition for African Languages](https://mlsanity.com/paper/2103.11811/) (2021)
- [PhoBERT: Pre-trained language models for Vietnamese](https://mlsanity.com/paper/2003.00744/) (2020)
- [Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning](https://mlsanity.com/paper/2204.06487/) (2022)
- [Multilingual and Multi-Aspect Hate Speech Analysis](https://mlsanity.com/paper/1908.11049/) (2019)
- [Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models](https://mlsanity.com/paper/2102.00894/) (2021)
- [A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation](https://mlsanity.com/paper/2205.02022/) (2022)
- [AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages](https://mlsanity.com/paper/2302.08956/) (2023)
- [NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysis](https://mlsanity.com/paper/2201.08277/) (2022)
- [AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languages](https://mlsanity.com/paper/2211.03263/) (2022)
- [SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)](https://mlsanity.com/paper/2304.06845/) (2023)
- [SERENGETI: Massively Multilingual Language Models for Africa](https://mlsanity.com/paper/2212.10785/) (2022)
- [MasakhaNEWS: News Topic Classification for African languages](https://mlsanity.com/paper/2304.09972/) (2023)
- [Introducing various Semantic Models for Amharic: Experimentation and Evaluation with multiple Tasks and Datasets](https://mlsanity.com/paper/2011.01154/) (2020)
- [Natural Language Processing in Ethiopian Languages: Current State, Challenges, and Opportunities](https://mlsanity.com/paper/2303.14406/) (2023)
- [An Amharic News Text classification Dataset](https://mlsanity.com/paper/2103.05639/) (2021)
- [Extended Parallel Corpus for Amharic-English Machine Translation](https://mlsanity.com/paper/2104.03543/) (2021)
- [NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis](https://mlsanity.com/paper/2305.00090/) (2023)
- [Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya](https://mlsanity.com/paper/2006.07698/) (2020)
- [ANEC: An Amharic Named Entity Corpus and Transformer Based Recognizer](https://mlsanity.com/paper/2207.00785/) (2022)

## Built on by (most cited first, 8 of 8)

- [Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding](https://mlsanity.com/paper/2412.17837/) (2024)
- [Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead](https://mlsanity.com/paper/2505.21315/) (2025)
- [Afrispeech-Dialog: A Benchmark Dataset for Spontaneous English Conversations in Healthcare and Beyond](https://mlsanity.com/paper/2502.03945/) (2025)
- [The State of Large Language Models for African Languages: Progress and Challenges](https://mlsanity.com/paper/2506.02280/) (2025)
- [Afri-MCQA: Multimodal Cultural Question Answering for African Languages](https://mlsanity.com/paper/2601.05699/) (2026)
- [Enhancing Multi-Label Emotion Analysis and Corresponding Intensities for Ethiopian Languages](https://mlsanity.com/paper/2503.18253/) (2025)
- [Amplify Initiative: Building A Localized Data Platform for Globalized AI](https://mlsanity.com/paper/2504.14105/) (2025)
- [CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems](https://mlsanity.com/paper/2609.22494/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have gained popularity recently due to their outstanding performance in various downstream Natural Language Processing (NLP) tasks. However, low-resource languages are still lagging behind current state-of-the-art (SOTA) developments in the field of NLP due to insufficient resources to train LLMs. Ethiopian languages exhibit remarkable linguistic diversity, encompassing a wide array of scripts, and are imbued with profound religious and cultural significance. This paper introduces EthioLLM -- multilingual large language models for five Ethiopian languages (Amharic, Ge'ez, Afan Oromo, Somali, and Tigrinya) and English, and Ethiobenchmark -- a new benchmark dataset for various downstream NLP tasks. We evaluate the performance of these models across five downstream NLP tasks. We open-source our multilingual language models, new benchmark datasets for various downstream tasks, and task-specific fine-tuned language models and discuss the performance of the models. Our dataset and models are available at the https://huggingface.co/EthioNLP repository.