mlsanityData to 14 Oct 2026

FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation

arXiv 2505.14256 ↗First posted 20 May 2025cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we present FuxiMT, a novel Chinese-centric multilingual machine translation model powered by a sparsified large language model (LLM). We adopt a two-stage strategy to train FuxiMT. We first pre-train the model on a massive Chinese corpus and then conduct multilingual fine-tuning on a large parallel dataset encompassing 65 languages. FuxiMT incorporates Mixture-of-Experts (MoEs) and employs a curriculum learning strategy for robust performance across various resource levels. Experimental results demonstrate that FuxiMT significantly outperforms strong baselines, including state-of-the-art LLMs and machine translation models, particularly under low-resource scenarios. Furthermore, FuxiMT exhibits remarkable zero-shot translation capabilities for unseen language pairs, indicating its potential to bridge communication gaps where parallel data are scarce or unavailable.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 42 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

25 of 42 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  9. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  10. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  11. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  12. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
13 more
  1. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  2. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  3. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  4. 2023BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languagescited by 60
  5. 2023PanGu-Sigma: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computingcited by 56
  6. 2023WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Modelscited by 50
  7. 2021Learning Language Specific Sub-network for Multilingual Machine Translationcited by 48
  8. 2024Fine-tuning Large Language Models for Domain-specific Machine Translationcited by 43
  9. 2021Counter-Interference Adapter for Multilingual Machine Translationcited by 40
  10. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  11. 2023ChineseWebText: Large-scale High-quality Chinese Web Text Extracted with Effective Evaluation Modelcited by 16
  12. 2024FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Datacited by 10
  13. 2023Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translationcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation
  2. 2026MERIT: Multilingual Expert-Reward Informed Tuning for Chinese-Centric Low-Resource Machine Translation
  3. 2026Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
  4. 2024Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning
  5. 2024Pruning Multilingual Large Language Models for Multilingual Inference
  6. 2026CAT-Translate: Building Compact Open-Source Models for Japanese-English Translation
  7. 2023The unreasonable effectiveness of few-shot learning for machine translation
  8. 2025The Uneven Impact of Post-Training Quantization in Machine Translation