# Federated Learning Of Out-Of-Vocabulary Words

arXiv: https://arxiv.org/abs/1903.10635 · HTML: https://mlsanity.com/paper/1903.10635/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2019.

## Built on by (most cited first, 30 of 51)

- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [FedScale: Benchmarking Model and System Performance of Federated Learning at Scale](https://mlsanity.com/paper/2105.11367/) (2021)
- [Federated Learning Meets Natural Language Processing: A Survey](https://mlsanity.com/paper/2107.12603/) (2021)
- [Communication Efficiency in Federated Learning: Achievements and Challenges](https://mlsanity.com/paper/2107.10996/) (2021)
- [Pretrained Models for Multilingual Federated Learning](https://mlsanity.com/paper/2206.02291/) (2022)
- [Can Public Large Language Models Help Private Cross-device Federated Learning?](https://mlsanity.com/paper/2305.12132/) (2023)
- [FedFace: Collaborative Learning of Face Recognition Model](https://mlsanity.com/paper/2104.03008/) (2021)
- [The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning](https://mlsanity.com/paper/2111.05326/) (2021)
- [FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMs](https://mlsanity.com/paper/2306.04959/) (2023)
- [NEO-BENCH: Evaluating Robustness of Large Language Models with Neologisms](https://mlsanity.com/paper/2402.12261/) (2024)
- [FLINT: A Platform for Federated Learning Integration](https://mlsanity.com/paper/2302.12862/) (2023)
- [Training Mixed-Domain Translation Models via Federated Learning](https://mlsanity.com/paper/2205.01557/) (2022)
- [Self-supervised Cross-silo Federated Neural Architecture Search](https://mlsanity.com/paper/2101.11896/) (2021)
- [SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learning](https://mlsanity.com/paper/2111.14345/) (2021)
- [Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data](https://mlsanity.com/paper/2405.07925/) (2024)
- [Enhancing Heterogeneous Federated Learning with Knowledge Extraction and Multi-Model Fusion](https://mlsanity.com/paper/2208.07978/) (2022)
- [Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies](https://mlsanity.com/paper/2109.03999/) (2021)
- [Federated Recommendation with Additive Personalization](https://mlsanity.com/paper/2301.09109/) (2023)
- [Pisces: Efficient Federated Learning via Guided Asynchronous Training](https://mlsanity.com/paper/2206.09264/) (2022)
- [FedTriNet: A Pseudo Labeling Method with Three Players for Federated Semi-supervised Learning](https://mlsanity.com/paper/2109.05612/) (2021)
- [Federated Online and Bandit Convex Optimization](https://mlsanity.com/paper/2311.17586/) (2023)
- [Monitoring and Adapting ML Models on Mobile Devices](https://mlsanity.com/paper/2305.07772/) (2023)
- [On the Byzantine-Resilience of Distillation-Based Federated Learning](https://mlsanity.com/paper/2402.12265/) (2024)
- [Heterogeneous Federated Learning using Dynamic Model Pruning and Adaptive Gradient](https://mlsanity.com/paper/2106.06921/) (2021)
- [Bristle: Decentralized Federated Learning in Byzantine, Non-i.i.d. Environments](https://mlsanity.com/paper/2110.11006/) (2021)
- [Multi-Model Federated Learning](https://mlsanity.com/paper/2201.02582/) (2022)
- [Training a Tokenizer for Free with Private Federated Learning](https://mlsanity.com/paper/2203.09943/) (2022)
- [Reconciling Security and Communication Efficiency in Federated Learning](https://mlsanity.com/paper/2207.12779/) (2022)
- [A Convergence Theory for Federated Average: Beyond Smoothness](https://mlsanity.com/paper/2211.01588/) (2022)
- [Confidential Federated Computations](https://mlsanity.com/paper/2404.10764/) (2024)

## Abstract (arXiv)

We demonstrate that a character-level recurrent neural network is able to learn out-of-vocabulary (OOV) words under federated learning settings, for the purpose of expanding the vocabulary of a virtual keyboard for smartphones without exporting sensitive text to servers. High-frequency words can be sampled from the trained generative model by drawing from the joint posterior directly. We study the feasibility of the approach in two settings: (1) using simulated federated learning on a publicly available non-IID per-user dataset from a popular social networking website, (2) using federated learning on data hosted on user mobile devices. The model achieves good recall and precision compared to ground-truth OOV words in setting (1). With (2) we demonstrate the practicality of this approach by showing that we can learn meaningful OOV words with good character-level prediction accuracy and cross entropy loss.