# Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation

arXiv: https://arxiv.org/abs/1809.03132 · HTML: https://mlsanity.com/paper/1809.03132/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2018.

## Built on by (most cited first, 9 of 9)

- [One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation](https://mlsanity.com/paper/2205.14333/) (2022)
- [Sequence-Level Training for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2106.08122/) (2021)
- [Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generation](https://mlsanity.com/paper/2106.15078/) (2021)
- [Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2210.03953/) (2022)
- [Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation](https://mlsanity.com/paper/2106.06751/) (2021)
- [Adaptive Bridge between Training and Inference for Dialogue](https://mlsanity.com/paper/2110.11560/) (2021)
- [N-gram Prediction and Word Difference Representations for Language Modeling](https://mlsanity.com/paper/2409.03295/) (2024)
- [Differentiable N-gram Objective on Abstractive Summarization](https://mlsanity.com/paper/2202.04003/) (2022)
- [N-Gram Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2301.12866/) (2023)

## Abstract (arXiv)

Neural machine translation (NMT) models are usually trained with the word-level loss using the teacher forcing algorithm, which not only evaluates the translation improperly but also suffers from exposure bias. Sequence-level training under the reinforcement framework can mitigate the problems of the word-level loss, but its performance is unstable due to the high variance of the gradient estimation. On these grounds, we present a method with a differentiable sequence-level training objective based on probabilistic n-gram matching which can avoid the reinforcement framework. In addition, this method performs greedy search in the training which uses the predicted words as context just as at inference to alleviate the problem of exposure bias. Experiment results on the NIST Chinese-to-English translation tasks show that our method significantly outperforms the reinforcement-based algorithms and achieves an improvement of 1.5 BLEU points on average over a strong baseline system.