mlsanityData to 14 Oct 2026

XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders

arXiv 2012.15547 ↗First posted 31 Dec 2020cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Multilingual machine translation enables a single model to translate between different languages. Most existing multilingual machine translation systems adopt a randomly initialized Transformer backbone. In this work, inspired by the recent success of language model pre-training, we present XLM-T, which initializes the model with an off-the-shelf pretrained cross-lingual Transformer encoder and fine-tunes it with multilingual parallel data. This simple method achieves significant improvements on a WMT dataset with 10 language pairs and the OPUS-100 corpus with 94 pairs. Surprisingly, the method is also effective even upon the strong baseline with back-translation. Moreover, extensive analysis of XLM-T on unsupervised syntactic parsing, word alignment, and multilingual classification explains its effectiveness for machine translation. The code will be at https://aka.ms/xlm-t.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 10; 2022: 6; 2023: 5; 2024: 3.

24 papers.

Sort:

By area:

  1. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
  2. 2022StableMoE: Stable Routing Strategy for Mixture of Expertscited by 100
  3. 2021DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoderscited by 50
  4. 2021A Primer on Pretrained Multilingual Language Modelscited by 49
  5. 2024xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoningcited by 46
  6. 2021MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairscited by 41
  7. 2021BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translationcited by 35
  8. 2021Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoderscited by 24
  9. 2021Unsupervised Neural Machine Translation with Generative Language Models Onlycited by 24
  10. 2021Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-trainingcited by 22
  11. 2022GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminatorcited by 20
  12. 2023mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learningcited by 14
12 more
  1. 2023FinPT: Financial Risk Prediction with Profile Tuning on Pretrained Foundation Modelscited by 13
  2. 2022A Frustratingly Simple Approach for End-to-End Image Captioningcited by 12
  3. 2021Multilingual Translation via Grafting Pre-trained Language Modelscited by 10
  4. 2022GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translationcited by 10
  5. 2023M2C: Towards Automatic Multimodal Manga Complementcited by 7
  6. 2024m3P: Towards Multimodal Multilingual Translation with Multimodal Promptcited by 6
  7. 2022HLT-MT: High-resource Language-specific Training for Multilingual Neural Machine Translationcited by 4
  8. 2023Core Building Blocks: Next Gen Geo Spatial GPT Applicationcited by 4
  9. 2022Multilingual Bidirectional Unsupervised Translation Through Multilingual Finetuning and Back-Translationcited by 3
  10. 2021Text Classification for Predicting Multi-level Product Categoriescited by 1
  11. 2024Enhancing Low-Resource NMT with a Multilingual Encoder and Knowledge Distillation: A Case Studycited by 1
  12. 2023Integrating Pre-trained Language Model into Neural Machine Translationcited by 0