mlsanityData to 14 Oct 2026

When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?

arXiv 1804.06323 ↗First posted 17 Apr 2018cs.CLcited by 126

Not on the map. Why?

Abstract (arXiv)

The performance of Neural Machine Translation (NMT) systems often suffers in low-resource scenarios where sufficiently large-scale parallel corpora cannot be obtained. Pre-trained word embeddings have proven to be invaluable for improving performance in natural language analysis tasks, which often suffer from paucity of data. However, their utility for NMT has not been extensively explored. In this work, we perform five sets of experiments that analyze when we can expect pre-trained word embeddings to help in NMT tasks. We show that such embeddings can be surprisingly effective in some cases -- providing gains of up to 20 BLEU points in the most favorable setting.

Key facts. Cited by 126 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 46; 2022: 26; 2023: 23; 2024: 20; 2025: 9; 2026: 2.

126 papers.

Sort:

By area:

  1. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  2. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  3. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
  4. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 181
  5. 2022DeepNet: Scaling Transformers to 1,000 Layerscited by 178
  6. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 109
  7. 2022mSLAM: Massively multilingual joint pre-training for speech and textcited by 108
  8. 2021Survey of Low-Resource Machine Translationcited by 92
  9. 2021Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translationcited by 74
  10. 2021IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generationcited by 57
  11. 2021A Survey on Green Deep Learningcited by 52
  12. 2023An Open Dataset and Model for Language Identificationcited by 50
38 more
  1. 2023GlotLID: Language Identification for Low-Resource Languagescited by 39
  2. 2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
  3. 2021When Does Translation Require Context? A Data-driven, Multilingual Explorationcited by 36
  4. 2021MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translatorscited by 35
  5. 2022Frustratingly Easy Label Projection for Cross-lingual Transfercited by 33
  6. 2023Prompt Optimization via Adversarial In-Context Learningcited by 33
  7. 2021Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitutioncited by 31
  8. 2021Many-to-English Machine Translation Tools, Data, and Pretrained Modelscited by 31
  9. 2022Motley: Benchmarking Heterogeneity and Personalization in Federated Learningcited by 31
  10. 2021Language Tags Matter for Zero-Shot Neural Machine Translationcited by 29
  11. 2021Data Augmentation for Sign Language Gloss Translationcited by 28
  12. 2022Using natural language prompts for machine translationcited by 27
  13. 2022RuCoLA: Russian Corpus of Linguistic Acceptabilitycited by 26
  14. 2023Matrix Compression via Randomized Low Rank and Low Precision Factorizationcited by 25
  15. 2021Multilingual Unsupervised Neural Machine Translation with Denoising Adapterscited by 22
  16. 2023Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfercited by 22
  17. 2021A Large-Scale Study of Machine Translation in the Turkic Languagescited by 21
  18. 2022Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translationcited by 20
  19. 2024ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Modelscited by 20
  20. 2021Distributionally Robust Multilingual Machine Translationcited by 18
  21. 2022Improving Word Translation via Two-Stage Contrastive Learningcited by 18
  22. 2024Examining Forgetting in Continual Pre-training of Aligned Large Language Modelscited by 18
  23. 2021Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Datacited by 17
  24. 2021Meta Back-translationcited by 16
  25. 2021Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Banditscited by 16
  26. 2021Robust Optimization for Multilingual Translation with Imbalanced Datacited by 15
  27. 2023Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lenscited by 15
  28. 2023M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Modelscited by 15
  29. 2022Language-Family Adapters for Low-Resource Multilingual Neural Machine Translationcited by 14
  30. 2025Krikri: Advancing Open Large Language Models for Greekcited by 14
  31. 2021Combining Static Word Embeddings and Contextual Representations for Bilingual Lexicon Inductioncited by 12
  32. 2021Continual Learning in Multilingual NMT via Language-Specific Embeddingscited by 12
  33. 2022You Are What You Write: Preserving Privacy in the Era of Large Language Modelscited by 12
  34. 2022Finding Memo: Extractive Memorization in Constrained Sequence Generation Taskscited by 12
  35. 2023How Vocabulary Sharing Facilitates Multilingualism in LLaMA?cited by 12
  36. 2021Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solutioncited by 11
  37. 2021Multilingual Translation via Grafting Pre-trained Language Modelscited by 10
  38. 2021Efficient Inference for Multilingual Neural Machine Translationcited by 10

50 most cited of 126 All 126 (JSON)