mlsanityData to 7 Oct 2026

Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter

arXiv 2305.12449 ↗First posted 21 May 2023cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Federated Multilingual Neural Machine Translation (Fed-MNMT) has emerged as a promising paradigm for institutions with limited language resources. This approach allows multiple institutions to act as clients and train a unified model through model synchronization, rather than collecting sensitive data for centralized training. This significantly reduces the cost of corpus collection and preserves data privacy. However, as pre-trained language models (PLMs) continue to increase in size, the communication cost for transmitting parameters during synchronization has become a training speed bottleneck. In this paper, we propose a communication-efficient Fed-MNMT framework that addresses this issue by keeping PLMs frozen and only transferring lightweight adapter modules between clients. Since different language pairs exhibit substantial discrepancies in data distributions, adapter parameters of clients may conflict with each other. To tackle this, we explore various clustering strategies to group parameters for integration and mitigate the negative effects of conflicting parameters. Experimental results demonstrate that our framework reduces communication cost by over 98% while achieving similar or even better performance compared to competitive baselines. Further analysis reveals that clustering strategies effectively solve the problem of linguistic discrepancy and pruning adapter modules further improves communication efficiency.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 31 of its 38 references that are in the map. First posted on arXiv 21 May 2023.

Builds on

31 of 38 references.

References · 31

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  4. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  5. 2018Federated Learning with Non-IID Datacited by 1,099
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  7. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  8. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  9. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  10. 2017Learning multiple visual domains with residual adapterscited by 553
  11. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  12. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
19 more
  1. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  2. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  3. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  4. 2019Massively Multilingual Neural Machine Translationcited by 246
  5. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  6. 2020AdapterDrop: On the Efficiency of Adapters in Transformerscited by 223
  7. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  8. 2021A Survey on Green Deep Learningcited by 52
  9. 2019Multilingual Neural Machine Translation with Language Clusteringcited by 47
  10. 2020CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascadecited by 47
  11. 2021Dynamic Knowledge Distillation for Pre-trained Language Modelscited by 38
  12. 2022Pretrained Models for Multilingual Federated Learningcited by 30
  13. 2020Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translationcited by 28
  14. 2020FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learningcited by 24
  15. 2021Federated Learning for Face Recognition with Gradient Correctioncited by 24
  16. 2021Multilingual Unsupervised Neural Machine Translation with Denoising Adapterscited by 22
  17. 2022Training Mixed-Domain Translation Models via Federated Learningcited by 12
  18. 2021Communication-Efficient Federated Learning for Neural Machine Translationcited by 5
  19. 2022A Federated Approach to Predicting Emojis in Hindi Tweetscited by 5

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 4; 2025: 3.

8 papers.

  1. 2024Advances and Open Challenges in Federated Foundation Modelscited by 33
  2. 2025A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learningcited by 15
  3. 2024Synergizing Foundation Models and Federated Learning: A Surveycited by 14
  4. 2023Tunable Soft Prompts are Messengers in Federated Learningcited by 10
  5. 2025Learning Like Humans: Resource-Efficient Federated Fine-Tuning through Cognitive Developmental Stagescited by 4
  6. 2025Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptationcited by 2
  7. 2024DAMe: Personalized Federated Social Event Detection with Dual Aggregation Mechanismcited by 1
  8. 2024FedMCP: Parameter-Efficient Federated Learning with Model-Contrastive Personalizationcited by 0