# A Federated Approach to Predicting Emojis in Hindi Tweets

arXiv: https://arxiv.org/abs/2211.06401 · HTML: https://mlsanity.com/paper/2211.06401/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 50 references that are in the map.
- First posted on arXiv 11 Nov 2022.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Applied Federated Learning: Improving Google Keyboard Query Suggestions](https://mlsanity.com/paper/1812.02903/) (2018)
- [Cost Sensitive Learning of Deep Feature Representations from Imbalanced Data](https://mlsanity.com/paper/1508.03422/) (2015)
- [Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm](https://mlsanity.com/paper/1708.00524/) (2017)
- [Federated Learning for Emoji Prediction in a Mobile Keyboard](https://mlsanity.com/paper/1906.04329/) (2019)
- [Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1512.05830/) (2015)
- [Deep Active Learning over the Long Tail](https://mlsanity.com/paper/1711.00941/) (2017)
- [Deep Imbalanced Attribute Classification using Visual Attention Aggregation](https://mlsanity.com/paper/1807.03903/) (2018)
- [Towards Causal Federated Learning For Enhanced Robustness and Privacy](https://mlsanity.com/paper/2104.06557/) (2021)
- [Emoji Prediction: Extensions and Benchmarking](https://mlsanity.com/paper/2007.07389/) (2020)
- [Contrastive Learning of Emoji-based Representations for Resource-Poor Languages](https://mlsanity.com/paper/1804.01855/) (2018)

## Built on by (most cited first, 5 of 5)

- [Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter](https://mlsanity.com/paper/2305.12449/) (2023)
- [Federated Learning Based Multilingual Emoji Prediction In Clean and Attack Scenarios](https://mlsanity.com/paper/2304.01005/) (2023)
- [On-Device Emoji Classifier Trained with GPT-based Data Augmentation for a Mobile Keyboard](https://mlsanity.com/paper/2411.05031/) (2024)
- [A Federated Approach for Hate Speech Detection](https://mlsanity.com/paper/2302.09243/) (2023)
- [FedMental: Evaluating Federated Learning for Mental Health Detection from Social Media Data](https://mlsanity.com/paper/2605.18936/) (2026)

## Abstract (arXiv)

The use of emojis affords a visual modality to, often private, textual communication. The task of predicting emojis however provides a challenge for machine learning as emoji use tends to cluster into the frequently used and the rarely used emojis. Much of the machine learning research on emoji use has focused on high resource languages and has conceptualised the task of predicting emojis around traditional server-side machine learning approaches. However, traditional machine learning approaches for private communication can introduce privacy concerns, as these approaches require all data to be transmitted to a central storage. In this paper, we seek to address the dual concerns of emphasising high resource languages for emoji prediction and risking the privacy of people's data. We introduce a new dataset of 118k tweets (augmented from 25k unique tweets) for emoji prediction in Hindi, and propose a modification to the federated learning algorithm, CausalFedGSD, which aims to strike a balance between model performance and user privacy. We show that our approach obtains comparative scores with more complex centralised models while reducing the amount of data required to optimise the models and minimising risks to user privacy.