mlsanityData to 14 Oct 2026

Detection of Somali-written Fake News and Toxic Messages on the Social Media Using Transformer-based Language Models

arXiv 2503.18117 ↗First posted 23 Mar 2025cs.CLcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

The fact that everyone with a social media account can create and share content, and the increasing public reliance on social media platforms as a news and information source bring about significant challenges such as misinformation, fake news, harmful content, etc. Although human content moderation may be useful to an extent and used by these platforms to flag posted materials, the use of AI models provides a more sustainable, scalable, and effective way to mitigate these harmful contents. However, low-resourced languages such as the Somali language face limitations in AI automation, including scarce annotated training datasets and lack of language models tailored to their unique linguistic characteristics. This paper presents part of our ongoing research work to bridge some of these gaps for the Somali language. In particular, we created two human-annotated social-media-sourced Somali datasets for two downstream applications, fake news & toxicity classification, and developed a transformer-based monolingual Somali language model (named SomBERTa) -- the first of its kind to the best of our knowledge. SomBERTa is then fine-tuned and evaluated on toxic content, fake news and news topic classification datasets. Comparative evaluation analysis of the proposed model against related multilingual models (e.g., AfriBERTa, AfroXLMR, etc) demonstrated that SomBERTa consistently outperformed these comparators in both fake news and toxic content classification tasks while achieving the best average accuracy (87.99%) across all tasks. This research contributes to Somali NLP by offering a foundational language model and a replicable framework for other low-resource languages, promoting digital and AI inclusivity and linguistic diversity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 19 of its 32 references that are in the map. First posted on arXiv 23 Mar 2025.

Builds on

19 of 32 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  7. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  8. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 362
  9. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 171
  10. 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
  11. 2022Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuningcited by 149
  12. 2022TimeLMs: Diachronic Language Models from Twittercited by 131
7 more
  1. 2020Extending Multilingual BERT to Low-Resource Languagescited by 88
  2. 2020Fast WordPiece Tokenizationcited by 87
  3. 2023Spanish Pre-trained BERT Model and Evaluation Datacited by 81
  4. 2017This Just In: Fake News Packs a Lot in Title, Uses Simpler, Repetitive Content in Text Body, More Similar to Satire than Real Newscited by 59
  5. 2022AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languagescited by 46
  6. 2021AI4D -- African Language Programcited by 18
  7. 2023Lexicon and Rule-based Word Lemmatization Approach for the Somali Languagecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Think Outside the Data: Colonial Biases and Systemic Issues in Automated Moderation Pipelines for Low-Resource Languages
  2. 2025Multilingual Hate Speech Detection in Social Media Using Translation-Based Approaches with Large Language Models
  3. 2025LLM-based Semantic Augmentation for Harmful Content Detection
  4. 2023"HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social media
  5. 2024Analysis and Detection of Multilingual Hate Speech Using Transformer Based Deep Learning
  6. 2025Rethinking Hate Speech Detection on Social Media: Can LLMs Replace Traditional Models?
  7. 2026DIA-HARM: Dialectal Disparities in Harmful Content Detection Across 50 English Dialects
  8. 2025Are LLMs Enough for Hyperpartisan, Fake, Polarized and Harmful Content Detection? Evaluating In-Context Learning vs. Fine-Tuning