mlsanityData to 7 Oct 2026

Importance-based Neuron Allocation for Multilingual Neural Machine Translation

arXiv 2107.06569 ↗First posted 14 Jul 2021cs.CLcited by 23

Not on the map. Why?

Abstract (arXiv)

Multilingual neural machine translation with a single model has drawn much attention due to its capability to deal with multiple languages. However, the current multilingual translation paradigm often makes the model tend to preserve the general knowledge, but ignore the language-specific knowledge. Some previous works try to solve this problem by adding various kinds of language-specific modules to the model, but they suffer from the parameter explosion problem and require specialized manual design. To solve these problems, we propose to divide the model neurons into general and language-specific parts based on their importance across languages. The general part is responsible for preserving the general knowledge and participating in the translation of all the languages, while the language-specific part is responsible for preserving the language-specific knowledge and participating in the translation of some specific languages. Experimental results on several language pairs, covering IWSLT and Europarl corpus datasets, demonstrate the effectiveness and universality of the proposed method.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 27 of its 35 references that are in the map. First posted on arXiv 14 Jul 2021.

Builds on

27 of 35 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  8. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  9. 2017Convolutional Sequence to Sequence Learningcited by 550
  10. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  11. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  12. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
15 more
  1. 2019Massively Multilingual Neural Machine Translationcited by 246
  2. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  3. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  4. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  5. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  6. 2018Universal Neural Machine Translation for Extremely Low Resource Languagescited by 66
  7. 2019Multilingual Neural Machine Translation with Language Clusteringcited by 47
  8. 2016Compression of Neural Machine Translation Models via Pruningcited by 43
  9. 2018Parameter Sharing Methods for Multilingual Self-Attentional Translation Modelscited by 33
  10. 2018Multilingual Neural Machine Translation with Task-Specific Attentioncited by 26
  11. 2020Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoderscited by 23
  12. 2018Multilingual NMT with a language-independent attention bridgecited by 20
  13. 2021Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translationcited by 15
  14. 2019Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networkscited by 12
  15. 2020On the Sparsity of Neural Machine Translation Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 2; 2024: 5; 2025: 4; 2026: 5.

23 papers.

Sort:

By area:

  1. 2023BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languagescited by 60
  2. 2024Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neuronscited by 23
  3. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  4. 2024Neuron Specialization: Leveraging intrinsic task modularity for multilingual machine translationcited by 19
  5. 2022Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translationcited by 18
  6. 2022CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translationcited by 17
  7. 2025Language-specific Neurons Do Not Facilitate Cross-Lingual Transfercited by 11
  8. 2026DNA: Uncovering Universal Latent Forgery Knowledgecited by 10
  9. 2022G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Taskscited by 9
  10. 2023Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learningcited by 9
  11. 2025Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generationcited by 8
  12. 2021Parameter Differentiation based Multilingual Neural Machine Translationcited by 5
11 more
  1. 2025How Programming Concepts and Neurons Are Shared in Code Language Modelscited by 4
  2. 2022Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiencycited by 3
  3. 2024Understanding the role of FFNs in driving multilingual behaviour in LLMscited by 3
  4. 2022Unifying the Convergences in Multilingual Neural Machine Translationcited by 2
  5. 2024Unraveling Babel: Exploring Multilingual Activation Patterns of LLMs and Their Applicationscited by 2
  6. 2026MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translationcited by 2
  7. 2022Informative Language Representation Learning for Massively Multilingual Neural Machine Translationcited by 1
  8. 2025Sparse Subnetwork Enhancement for Underrepresented Languages in Large Language Modelscited by 1
  9. 2024Effectively Prompting Small-sized Language Models for Cross-lingual Tasks via Winning Ticketscited by 0
  10. 2026Consensus-Aligned Neuron Efficient Fine-Tuning Large Language Models for Multi-Domain Machine Translationcited by 0
  11. 2026V-FIND: Revealing the Intrinsic Forgery Knowledge Encoded in Video Forgery Detectorscited by 0