mlsanityData to 14 Oct 2026

Training Mixed-Domain Translation Models via Federated Learning

arXiv 2205.01557 ↗First posted 3 May 2022cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Training mixed-domain translation models is a complex task that demands tailored architectures and costly data preparation techniques. In this work, we leverage federated learning (FL) in order to tackle the problem. Our investigation demonstrates that with slight modifications in the training process, neural machine translation (NMT) engines can be easily adapted when an FL-based aggregation is applied to fuse different domains. Experimental results also show that engines built via FL are able to perform on par with state-of-the-art baselines that rely on centralized training techniques. We evaluate our hypothesis in the presence of five datasets with different sizes, from different domains, to translate from German into English and discuss how FL and NMT can mutually benefit from each other. In addition to providing benchmarking results on the union of FL and NMT, we also propose a novel technique to dynamically control the communication bandwidth by selecting impactful parameters during FL updates. This is a significant achievement considering the large size of NMT engines that need to be exchanged between FL parties.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 35 of its 42 references that are in the map. First posted on arXiv 3 May 2022.

Builds on

35 of 42 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  7. 2018Deep contextualized word representationscited by 2,290
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  9. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  10. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  11. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  12. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
23 more
  1. 2017Convolutional Sequence to Sequence Learningcited by 552
  2. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
  3. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
  4. 2020AdapterHub: A Framework for Adapting Transformerscited by 444
  5. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  6. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 265
  7. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 242
  8. 2020AdapterDrop: On the Efficiency of Adapters in Transformerscited by 224
  9. 2018Federated Learning for Keyword Spottingcited by 95
  10. 2018A Survey of Domain Adaptation for Neural Machine Translationcited by 64
  11. 2019Federated Learning Of Out-Of-Vocabulary Wordscited by 51
  12. 2018MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasetscited by 49
  13. 2018Learning Private Neural Language Modeling with Attentive Aggregationcited by 49
  14. 2021Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementationcited by 40
  15. 2021GRNN: Generative Regression Neural Network -- A Data Leakage Attack for Federated Learningcited by 36
  16. 2019Federated User Representation Learningcited by 32
  17. 2019Federated Learning for Ranking Browser History Suggestionscited by 24
  18. 2020FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learningcited by 24
  19. 2020Pretraining Federated Text Models for Next Word Predictioncited by 22
  20. 2020Federated pretraining and fine tuning of BERT using clinical notes from multiple siloscited by 18
  21. 2020Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learningcited by 16
  22. 2020Revisiting Robust Neural Machine Translation: A Transformer Case Studycited by 7
  23. 2021Communication-Efficient Federated Learning for Neural Machine Translationcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 8; 2024: 2; 2025: 1.

12 papers.

  1. 2023Tunable Soft Prompts are Messengers in Federated Learningcited by 10
  2. 2023Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adaptercited by 8
  3. 2023DP-NMT: Scalable Differentially-Private Machine Translationcited by 7
  4. 2023Federated Nearest Neighbor Machine Translationcited by 5
  5. 2022Collaborating Heterogeneous Natural Language Processing Tasks via Federated Learningcited by 4
  6. 2023Federated Learning Based Multilingual Emoji Prediction In Clean and Attack Scenarioscited by 3
  7. 2024Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translationcited by 2
  8. 2023Personalized Federated Learning via Gradient Modulation for Heterogeneous Text Summarizationcited by 1
  9. 2023A Comprehensive View of Personalized Federated Learning on Heterogeneous Clinical Datasetscited by 1
  10. 2023Fingerprint Attack: Client De-Anonymization in Federated Learningcited by 1
  11. 2024FedMCP: Parameter-Efficient Federated Learning with Model-Contrastive Personalizationcited by 0
  12. 2025Adaptive Latent-Space Constraints in Personalized Federated Learningcited by 0