mlsanityData to 14 Oct 2026

Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation

arXiv 2212.10551 ↗First posted 20 Dec 2022cs.CL cs.AIcited by 21

Not on the map. Why?

Abstract (arXiv)

Multilingual neural machine translation (MNMT) aims to build a unified model for many language directions. Existing monolithic models for MNMT encounter two challenges: parameter interference among languages and inefficient inference for large models. In this paper, we revisit the classic multi-way structures and develop a detachable model by assigning each language (or group of languages) to an individual branch that supports plug-and-play training and inference. To address the needs of learning representations for all languages in a unified space, we propose a novel efficient training recipe, upon which we build an effective detachable model, Lego-MT. For a fair comparison, we collect data from OPUS and build a translation benchmark covering 433 languages and 1.3B parallel data. Experiments show that Lego-MT with 1.2B parameters brings an average gain of 3.2 spBLEU. It even outperforms M2M-100 with 12B parameters. The proposed training recipe brings a 28.2× speedup over the conventional multi-way training method.footnote urlhttps://github.com/CONE-MT/Lego-MT.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

29 of 38 references.

References · 29

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  3. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  4. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  5. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  6. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  7. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  8. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  9. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  10. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  11. 2019Massively Multilingual Neural Machine Translationcited by 246
  12. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 242
17 more
  1. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  2. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  3. 2022StableMoE: Stable Routing Strategy for Mixture of Expertscited by 100
  4. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  5. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  6. 2018Universal Neural Machine Translation for Extremely Low Resource Languagescited by 66
  7. 2020Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Informationcited by 64
  8. 2016Fully Character-Level Neural Machine Translation without Explicit Segmentationcited by 55
  9. 2021Learning Language Specific Sub-network for Multilingual Machine Translationcited by 48
  10. 2021Counter-Interference Adapter for Multilingual Machine Translationcited by 40
  11. 2016Multi-Source Neural Translationcited by 37
  12. 2018Parameter Sharing Methods for Multilingual Self-Attentional Translation Modelscited by 33
  13. 2019Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translationcited by 23
  14. 2021Robust Optimization for Multilingual Translation with Imbalanced Datacited by 15
  15. 2021Multilingual Translation via Grafting Pre-trained Language Modelscited by 10
  16. 2022LegoNN: Building Modular Encoder-Decoder Modelscited by 9
  17. 2020Revisiting Modularized Multilingual NMT to Meet Industrial Demandscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 9; 2025: 7; 2026: 3.

21 papers.

Sort:

By area:

  1. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  2. 2024LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languagescited by 48
  3. 2023Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Modelscited by 39
  4. 2025A Survey of Multilingual Reasoning in Language Modelscited by 33
  5. 2023How Vocabulary Sharing Facilitates Multilingualism in LLaMA?cited by 12
  6. 2024Hire a Linguist!: Learning Endangered Languages with In-Context Linguistic Descriptionscited by 12
  7. 2025AFRIDOC-MT: Document-level MT Corpus for African Languagescited by 11
  8. 2024Krey`ol-MT: Building MT for Latin American, Caribbean and Colonial African Creole Languagescited by 10
  9. 2025Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resourcescited by 10
  10. 2024Let's Focus on Neuron: Neuron-Level Supervised Fine-tuning for Large Language Modelcited by 8
  11. 2025Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenoncited by 5
  12. 2025Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Datacited by 4
9 more
  1. 2024KS-Lottery: Finding Certified Lottery Tickets for Multilingual Language Modelscited by 3
  2. 2024Asynchronous and Segmented Bidirectional Encoding for NMTcited by 2
  3. 2024MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinkicited by 2
  4. 2024Pula: Training Large Language Models for Setswanacited by 2
  5. 2025EnAnchored-X2X: English-Anchored Optimization for Many-to-Many Translationcited by 2
  6. 2025Limited-Resource Adapters Are Regularizers, Not Linguistscited by 0
  7. 2026Enhancing Multilingual Reasoning via Steerable Model Mergingcited by 0
  8. 2026Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Modelscited by 0
  9. 2026Training Specialist Models without Reasoning Trajectories for Domain Expert Distillationcited by 0