mlsanityData to 14 Oct 2026

Federated Learning for Emoji Prediction in a Mobile Keyboard

arXiv 1906.04329 ↗First posted 11 Jun 2019cs.CL cs.LGcited by 118

Not on the map. Why?

Abstract (arXiv)

We show that a word-level recurrent neural network can predict emoji from text typed on a mobile keyboard. We demonstrate the usefulness of transfer learning for predicting emoji by pretraining the model using a language modeling task. We also propose mechanisms to trigger emoji and tune the diversity of candidates. The model is trained using a distributed on-device learning framework called federated learning. The federated model is shown to achieve better performance than a server-trained model. This work demonstrates the feasibility of using federated learning to train production-quality models for natural language understanding tasks while keeping users' data on their devices.

Key facts. Cited by 118 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 31; 2022: 27; 2023: 21; 2024: 27; 2025: 5; 2026: 7.

118 papers.

Sort:

By area:

  1. 2021A Field Guide to Federated Optimizationcited by 315
  2. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  3. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 128
  4. 2022Federated Class-Incremental Learningcited by 127
  5. 2021FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Taskscited by 85
  6. 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 68
  7. 2022LAMP: Extracting Text from Gradients with Language Model Priorscited by 54
  8. 2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Modelscited by 53
  9. 2023No One Left Behind: Real-World Federated Class-Incremental Learningcited by 53
  10. 2021Federated Learning Meets Natural Language Processing: A Surveycited by 43
  11. 2021Communication Efficiency in Federated Learning: Achievements and Challengescited by 40
  12. 2022Pretrained Models for Multilingual Federated Learningcited by 30
38 more
  1. 2022A Critical Review on the Use (and Misuse) of Differential Privacy in Machine Learningcited by 27
  2. 2021FedFace: Collaborative Learning of Face Recognition Modelcited by 26
  3. 2021RoFL: Robustness of Secure Federated Learningcited by 25
  4. 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
  5. 2021GIFAIR-FL: A Framework for Group and Individual Fairness in Federated Learningcited by 21
  6. 2023Privacy and Efficiency of Communications in Federated Split Learningcited by 21
  7. 2023FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMscited by 21
  8. 2024Advances in APPFL: A Comprehensive and Extensible Federated Learning Frameworkcited by 19
  9. 2021FedAR: Activity and Resource-Aware Federated Learning Model for Distributed Mobile Robotscited by 18
  10. 2023FLINT: A Platform for Federated Learning Integrationcited by 17
  11. 2022Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devicescited by 16
  12. 2021Real-time End-to-End Federated Learning: An Automotive Case Studycited by 15
  13. 2024Federated Learning in Practice: Reflections and Projectionscited by 15
  14. 2021Faster Rates for Compressed Federated Learning with Client-Variance Reductioncited by 14
  15. 2023Mutual Enhancement of Large and Small Language Models with Cross-Silo Knowledge Transfercited by 13
  16. 2024Personalized Federated Learning via Feature Distribution Adaptationcited by 13
  17. 2021Enabling On-Device Training of Speech Recognition Models with Federated Dropoutcited by 12
  18. 2024Fed3DGS: Scalable 3D Gaussian Splatting with Federated Learningcited by 12
  19. 2021Self-supervised Cross-silo Federated Neural Architecture Searchcited by 11
  20. 2021Opportunistic Federated Learning: An Exploration of Egocentric Collaboration for Pervasive Computing Applicationscited by 11
  21. 2022FL_PyTorch: optimization research simulator for federated learningcited by 10
  22. 2024Optimal Federated Learning for Nonparametric Regression with Heterogeneous Distributed Differential Privacy Constraintscited by 10
  23. 2021FL-Market: Trading Private Models in Federated Learningcited by 9
  24. 2021Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategiescited by 9
  25. 2023ULDP-FL: Federated Learning with Across Silo User-Level Differential Privacycited by 9
  26. 2024Analysis of Privacy Leakage in Federated Large Language Modelscited by 9
  27. 2024Privacy Preserving Federated Learning in Medical Imaging with Uncertainty Estimationcited by 9
  28. 2024Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailabilitycited by 9
  29. 2022Mixed Federated Learning: Joint Decentralized and Centralized Learningcited by 8
  30. 2022Pisces: Efficient Federated Learning via Guided Asynchronous Trainingcited by 8
  31. 2022Byzantines can also Learn from History: Fall of Centered Clipping in Federated Learningcited by 8
  32. 2021Fidel: Reconstructing Private Training Samples from Weight Updates in Federated Learningcited by 7
  33. 2021Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shiftcited by 7
  34. 2024MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classescited by 7
  35. 2022Federated Domain Adaptation for ASR with Full Self-Supervisioncited by 6
  36. 2023Monitoring and Adapting ML Models on Mobile Devicescited by 6
  37. 2023Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learningcited by 6
  38. 2024Federated Nonparametric Hypothesis Testing with Differential Privacy Constraints: Optimal Rates and Adaptive Testscited by 6

50 most cited of 118 All 118 (JSON)