# Federated Learning for Emoji Prediction in a Mobile Keyboard

arXiv: https://arxiv.org/abs/1906.04329 · HTML: https://mlsanity.com/paper/1906.04329/

## Key facts

- Cited by 118 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2019.

## Built on by (most cited first, 30 of 118)

- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Sustainable AI: Environmental Implications, Challenges and Opportunities](https://mlsanity.com/paper/2111.00364/) (2021)
- [The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation](https://mlsanity.com/paper/2102.06387/) (2021)
- [Federated Class-Incremental Learning](https://mlsanity.com/paper/2203.11473/) (2022)
- [FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks](https://mlsanity.com/paper/2104.08815/) (2021)
- [The Skellam Mechanism for Differentially Private Federated Learning](https://mlsanity.com/paper/2110.04995/) (2021)
- [LAMP: Extracting Text from Gradients with Language Model Priors](https://mlsanity.com/paper/2202.08827/) (2022)
- [Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Models](https://mlsanity.com/paper/2201.12675/) (2022)
- [No One Left Behind: Real-World Federated Class-Incremental Learning](https://mlsanity.com/paper/2302.00903/) (2023)
- [Federated Learning Meets Natural Language Processing: A Survey](https://mlsanity.com/paper/2107.12603/) (2021)
- [Communication Efficiency in Federated Learning: Achievements and Challenges](https://mlsanity.com/paper/2107.10996/) (2021)
- [Pretrained Models for Multilingual Federated Learning](https://mlsanity.com/paper/2206.02291/) (2022)
- [A Critical Review on the Use (and Misuse) of Differential Privacy in Machine Learning](https://mlsanity.com/paper/2206.04621/) (2022)
- [FedFace: Collaborative Learning of Face Recognition Model](https://mlsanity.com/paper/2104.03008/) (2021)
- [RoFL: Robustness of Secure Federated Learning](https://mlsanity.com/paper/2107.03311/) (2021)
- [The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning](https://mlsanity.com/paper/2111.05326/) (2021)
- [GIFAIR-FL: A Framework for Group and Individual Fairness in Federated Learning](https://mlsanity.com/paper/2108.02741/) (2021)
- [Privacy and Efficiency of Communications in Federated Split Learning](https://mlsanity.com/paper/2301.01824/) (2023)
- [FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMs](https://mlsanity.com/paper/2306.04959/) (2023)
- [Advances in APPFL: A Comprehensive and Extensible Federated Learning Framework](https://mlsanity.com/paper/2409.11585/) (2024)
- [FedAR: Activity and Resource-Aware Federated Learning Model for Distributed Mobile Robots](https://mlsanity.com/paper/2101.03705/) (2021)
- [FLINT: A Platform for Federated Learning Integration](https://mlsanity.com/paper/2302.12862/) (2023)
- [Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices](https://mlsanity.com/paper/2207.08988/) (2022)
- [Real-time End-to-End Federated Learning: An Automotive Case Study](https://mlsanity.com/paper/2103.11879/) (2021)
- [Federated Learning in Practice: Reflections and Projections](https://mlsanity.com/paper/2410.08892/) (2024)
- [Faster Rates for Compressed Federated Learning with Client-Variance Reduction](https://mlsanity.com/paper/2112.13097/) (2021)
- [Mutual Enhancement of Large and Small Language Models with Cross-Silo Knowledge Transfer](https://mlsanity.com/paper/2312.05842/) (2023)
- [Personalized Federated Learning via Feature Distribution Adaptation](https://mlsanity.com/paper/2411.00329/) (2024)
- [Enabling On-Device Training of Speech Recognition Models with Federated Dropout](https://mlsanity.com/paper/2110.03634/) (2021)
- [Fed3DGS: Scalable 3D Gaussian Splatting with Federated Learning](https://mlsanity.com/paper/2403.11460/) (2024)

## Abstract (arXiv)

We show that a word-level recurrent neural network can predict emoji from text typed on a mobile keyboard. We demonstrate the usefulness of transfer learning for predicting emoji by pretraining the model using a language modeling task. We also propose mechanisms to trigger emoji and tune the diversity of candidates. The model is trained using a distributed on-device learning framework called federated learning. The federated model is shown to achieve better performance than a server-trained model. This work demonstrates the feasibility of using federated learning to train production-quality models for natural language understanding tasks while keeping users' data on their devices.