Language-Informed Beam Search Decoding for Multilingual Machine Translation
Not on the map. Why?
Abstract (arXiv)
Beam search decoding is the de-facto method for decoding auto-regressive Neural Machine Translation (NMT) models, including multilingual NMT where the target language is specified as an input. However, decoding multilingual NMT models commonly produces ``off-target'' translations -- yielding translation outputs not in the intended language. In this paper, we first conduct an error analysis of off-target translations for a strong multilingual NMT model and identify how these decodings are produced during beam search. We then propose Language-informed Beam Search (LiBS), a general decoding algorithm incorporating an off-the-shelf Language Identification (LiD) model into beam search decoding to reduce off-target translations. LiBS is an inference-time procedure that is NMT-model agnostic and does not require any additional parallel data. Results show that our proposed LiBS algorithm on average improves +1.1 BLEU and +0.9 BLEU on WMT and OPUS datasets, and reduces off-target rates from 22.9% to 7.7% and 65.8% to 25.3% respectively.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 23 of its 26 references that are in the map. First posted on arXiv 11 Aug 2024.
Builds on
23 of 26 references.
References · 23
- 2017Attention Is All You Needcited by 60,361
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
- 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
- 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
- 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
- 2016FastText.zip: Compressing text classification modelscited by 335
- 2017Six Challenges for Neural Machine Translationcited by 335
- 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
11 more
- 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 138
- 2018Analyzing Uncertainty in Neural Machine Translationcited by 126
- 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
- 2019Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlationscited by 61
- 2019The Missing Ingredient in Zero-Shot Neural Machine Translationcited by 59
- 2019Improving Deep Transformer with Depth-Scaled Initialization and Merged Attentioncited by 59
- 2020Multi-task Learning for Multilingual Neural Machine Translationcited by 46
- 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42
- 2017Deep Reinforcement Learning-based Image Captioning with Embedding Rewardcited by 40
- 2021Improving Multilingual Translation by Representation and Gradient Regularizationcited by 31
- 2020Subword Segmentation and a Single Bridge Language Affect Zero-Shot Neural Machine Translationcited by 13
Built on by
Citing papers per year: 2025: 2.
2 papers.