Improving Lexically Constrained Neural Machine Translation with Source-Conditioned Masked Span Prediction
Not on the map. Why?
Abstract (arXiv)
Accurate terminology translation is crucial for ensuring the practicality and reliability of neural machine translation (NMT) systems. To address this, lexically constrained NMT explores various methods to ensure pre-specified words and phrases appear in the translation output. However, in many cases, those methods are studied on general domain corpora, where the terms are mostly uni- and bi-grams (>98%). In this paper, we instead tackle a more challenging setup consisting of domain-specific corpora with much longer n-gram and highly specialized terms. Inspired by the recent success of masked span prediction models, we propose a simple and effective training strategy that achieves consistent improvements on both terminology and sentence-level translation for three domain-specific corpora in two language pairs.
Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Builds on 29 of its 43 references that are in the map. First posted on arXiv 12 May 2021.
Builds on
29 of 43 references.
References · 29
- 2017Attention Is All You Needcited by 60,106
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,884
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,910
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,793
- 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,487
- 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,473
- 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 833
- 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
- 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
- 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 346
17 more
- 2017Six Challenges for Neural Machine Translationcited by 335
- 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 234
- 2019Levenshtein Transformercited by 198
- 2018Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translationcited by 181
- 2020Unsupervised Domain Clusters in Pretrained Language Modelscited by 160
- 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
- 2019Training Neural Machine Translation To Apply Terminology Constraintscited by 73
- 2019Domain Robustness in Neural Machine Translationcited by 65
- 2018A Survey of Domain Adaptation for Neural Machine Translationcited by 64
- 2019Code-Switching for Enhancing NMT with Pre-Specified Translationcited by 53
- 2020Lexically Constrained Neural Machine Translation with Levenshtein Transformercited by 45
- 2018Neural Machine Translation Decoding with Terminology Constraintscited by 36
- 2019Domain Adaptation of Neural Machine Translation by Lexicon Inductioncited by 33
- 2016SYSTRAN's Pure Neural Machine Translation Systemscited by 24
- 2020Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Modelcited by 24
- 2016Bridging Neural Machine Translation and Bilingual Dictionariescited by 18
- 2020Dictionary-based Data Augmentation for Cross-Domain Neural Machine Translationcited by 13
Built on by
Citing papers per year: 2022: 2; 2023: 1.
3 papers.