# Machine learning and emoji prediction: How much accuracy can MARBERT achieve?

arXiv: https://arxiv.org/abs/2604.21108 · HTML: https://mlsanity.com/paper/2604.21108/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 22 Apr 2026.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic](https://mlsanity.com/paper/2101.01785/) (2020)
- [AraGPT2: Pre-Trained Transformer for Arabic Language Generation](https://mlsanity.com/paper/2012.15520/) (2020)
- [Decoding ChatGPT: A Taxonomy of Existing Research, Current Challenges, and Possible Future Directions](https://mlsanity.com/paper/2307.14107/) (2023)
- [Generative linguistics contribution to artificial intelligence: Where this contribution lies?](https://mlsanity.com/paper/2410.20221/) (2024)
- [What fifty-one years of Linguistics and Artificial Intelligence research tell us about their correlation: A scientometric analysis](https://mlsanity.com/paper/2411.19858/) (2024)
- [Are Emojis Predictable?](https://mlsanity.com/paper/1702.07285/) (2017)
- [A Web Scraping Methodology for Bypassing Twitter API Restrictions](https://mlsanity.com/paper/1803.09875/) (2018)
- [A `Sourceful' Twist: Emoji Prediction Based on Sentiment, Hashtags and Application Source](https://mlsanity.com/paper/2103.07833/) (2021)
- [Emoji Prediction in Tweets using BERT](https://mlsanity.com/paper/2307.02054/) (2023)

## Built on by (most cited first, 2 of 2)

- [Does Machine "know" interpersonal pragmatics? Evidence from MARBERT's learning of emoji pragmatics in Arabic digital discourse](https://mlsanity.com/paper/2608.01174/) (2026)
- [Domain-specific Pretraining Profile and Transformer Performance: Evidence from Modeling Digital Pragmatics in Arabic-English Code-switching](https://mlsanity.com/paper/2609.14571/) (2026)

## Abstract (arXiv)

This study investigates Machine Learning (ML) in the prediction of emojis in Arabic tweets employing the (state-of-the-art) MARBERT model. A corpus of 11379 CA tweets representing multiple Arabic colloquial dialects was collected from X.com via Python. A net dataset includes 8695 tweets, which were utilized for the analysis. These tweets were then classified into 14 categories, which were numerically encoded and used as labels. A preprocessing pipeline was designed as an interpretable baseline, allowing us to examine the relationship between lexical features and emoji categories. MARBERT was finetuned to predict emoji use from textual input. We evaluated the model performance in terms of precision, recall and F1-scores. Findings reveal that the model performed quite well with an overall accuracy 0.75. The study concludes that although the findings are promising, there is still a need for improving machine learning models including MARBERT, specifically for low-resource and multidialectal languages like Arabic.