# Bag-of-Words as Target for Neural Machine Translation

arXiv: https://arxiv.org/abs/1805.04871 · HTML: https://mlsanity.com/paper/1805.04871/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 May 2018.

## Built on by (most cited first, 11 of 11)

- [How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources](https://mlsanity.com/paper/2306.04751/) (2023)
- [Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2106.05093/) (2021)
- [BERTTune: Fine-Tuning Neural Machine Translation with BERTScore](https://mlsanity.com/paper/2106.02208/) (2021)
- [Sequence-Level Training for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2106.08122/) (2021)
- [UniKeyphrase: A Unified Extraction and Generation Framework for Keyphrase Prediction](https://mlsanity.com/paper/2106.04847/) (2021)
- [KECRS: Towards Knowledge-Enriched Conversational Recommendation System](https://mlsanity.com/paper/2105.08261/) (2021)
- [Jam or Cream First? Modeling Ambiguity in Neural Machine Translation with SCONES](https://mlsanity.com/paper/2205.00704/) (2022)
- [ngram-OAXE: Phrase-Based Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2210.03999/) (2022)
- [N-gram Prediction and Word Difference Representations for Language Modeling](https://mlsanity.com/paper/2409.03295/) (2024)
- [Differentiable N-gram Objective on Abstractive Summarization](https://mlsanity.com/paper/2202.04003/) (2022)
- [Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers](https://mlsanity.com/paper/2605.07772/) (2026)

## Abstract (arXiv)

A sentence can be translated into more than one correct sentences. However, most of the existing neural machine translation models only use one of the correct translations as the targets, and the other correct sentences are punished as the incorrect sentences in the training stage. Since most of the correct translations for one sentence share the similar bag-of-words, it is possible to distinguish the correct translations from the incorrect ones by the bag-of-words. In this paper, we propose an approach that uses both the sentences and the bag-of-words as targets in the training stage, in order to encourage the model to generate the potentially correct sentences that are not appeared in the training set. We evaluate our model on a Chinese-English translation dataset, and experiments show our model outperforms the strong baselines by the BLEU score of 4.55.