mlsanityData to 14 Oct 2026

On the use of BERT for Neural Machine Translation

arXiv 1909.12744 ↗First posted 27 Sep 2019cs.CL cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Exploiting large pretrained models for various NMT tasks have gained a lot of visibility recently. In this work we study how BERT pretrained models could be exploited for supervised Neural Machine Translation. We compare various ways to integrate pretrained BERT model with NMT model and study the impact of the monolingual data used for BERT training on the final translation quality. We use WMT-14 English-German, IWSLT15 English-German and IWSLT14 English-Russian datasets for these experiments. In addition to standard task test set evaluation, we perform evaluation on out-of-domain test sets and noise injected test sets, in order to assess how BERT pretrained representations affect model robustness.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 11; 2022: 5; 2023: 3; 2024: 2; 2025: 2.

23 papers.

Sort:

  1. 2021CURE: Code-Aware Neural Machine Translation for Automatic Program Repaircited by 86
  2. 2021From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understandingcited by 43
  3. 2021BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translationcited by 35
  4. 2021Prevent the Language Model from being Overconfident in Neural Machine Translationcited by 34
  5. 2025Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptationcited by 28
  6. 2021BERTTune: Fine-Tuning Neural Machine Translation with BERTScorecited by 18
  7. 2022Changing the Representation: Examining Language Representation for Neural Sign Language Productioncited by 15
  8. 2023Implementing BERT and fine-tuned RobertA to detect AI generated news by ChatGPTcited by 13
  9. 2021Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviewscited by 7
  10. 2022Conditional Bilingual Mutual Information Based Adaptive Training for Neural Machine Translationcited by 6
  11. 2022Artificial intelligence for topic modelling in Hindu philosophy: mapping themes between the Upanishads and the Bhagavad Gitacited by 6
  12. 2021Better Neural Machine Translation by Extracting Linguistic Information from BERTcited by 5
11 more
  1. 2024Investigating Neural Machine Translation for Low-Resource Languages: Using Bavarian as a Case Studycited by 5
  2. 2021BERTGEN: Multi-task Generation through BERTcited by 3
  3. 2022Checks and Strategies for Enabling Code-Switched Machine Translationcited by 3
  4. 2023An Experimental Study on Pretraining Transformers from Scratch for IRcited by 3
  5. 2024FOCUS: Forging Originality through Contrastive Use in Self-Plagiarism for Language Modelscited by 3
  6. 2025From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development -- An Opinion Papercited by 3
  7. 2021EdinSaar@WMT21: North-Germanic Low-Resource Multilingual NMTcited by 2
  8. 2021Android Security using NLP Techniques: A Reviewcited by 0
  9. 2021Masked Adversarial Generation for Neural Machine Translationcited by 0
  10. 2022Optimizing Deep Transformers for Chinese-Thai Low-Resource Translationcited by 0
  11. 2023Integrating Pre-trained Language Model into Neural Machine Translationcited by 0