# Mixup Decoding for Diverse Machine Translation

arXiv: https://arxiv.org/abs/2109.03402 · HTML: https://mlsanity.com/paper/2109.03402/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 20 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [Scaling Neural Machine Translation](https://mlsanity.com/paper/1806.00187/) (2018)
- [Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement](https://mlsanity.com/paper/1802.06901/) (2018)
- [Mixture Models for Diverse Machine Translation: Tricks of the Trade](https://mlsanity.com/paper/1902.07816/) (2019)
- [Edinburgh Neural Machine Translation Systems for WMT 16](https://mlsanity.com/paper/1606.02891/) (2016)
- [Sequence-Level Mixed Sample Data Augmentation](https://mlsanity.com/paper/2011.09039/) (2020)
- [A Simple, Fast Diverse Decoding Algorithm for Neural Generation](https://mlsanity.com/paper/1611.08562/) (2016)
- [Lost in Translation: Loss and Decay of Linguistic Richness in Machine Translation](https://mlsanity.com/paper/1906.12068/) (2019)
- [Token-level Adaptive Training for Neural Machine Translation](https://mlsanity.com/paper/2010.04380/) (2020)
- [Generating Diverse Translation by Manipulating Multi-Head Attention](https://mlsanity.com/paper/1911.09333/) (2019)
- [Sequence to Sequence Mixture Model for Diverse Machine Translation](https://mlsanity.com/paper/1810.07391/) (2018)
- [Generating Diverse Translation from Model Distribution with Dropout](https://mlsanity.com/paper/2010.08178/) (2020)

## Built on by (most cited first, 10 of 10)

- [STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation](https://mlsanity.com/paper/2203.10426/) (2022)
- [EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling](https://mlsanity.com/paper/2403.14541/) (2024)
- [A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability](https://mlsanity.com/paper/2212.10888/) (2022)
- [Neural Machine Translation with Phrase-Level Universal Visual Representations](https://mlsanity.com/paper/2203.10299/) (2022)
- [One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation](https://mlsanity.com/paper/2205.14333/) (2022)
- [A Survey on Mixup Augmentations and Beyond](https://mlsanity.com/paper/2409.05202/) (2024)
- [To Diverge or Not to Diverge: A Morphosyntactic Perspective on Machine Translation vs Human Translation](https://mlsanity.com/paper/2401.01419/) (2024)
- [Exploring Diversity in Back Translation for Low-Resource Machine Translation](https://mlsanity.com/paper/2206.00564/) (2022)
- [For Better or For Worse? Learning Minimum Variance Features With Label Augmentation](https://mlsanity.com/paper/2402.06855/) (2024)
- [Generating Diverse Translation with Perturbed kNN-MT](https://mlsanity.com/paper/2402.09344/) (2024)

## Abstract (arXiv)

Diverse machine translation aims at generating various target language translations for a given source language sentence. Leveraging the linear relationship in the sentence latent space introduced by the mixup training, we propose a novel method, MixDiversity, to generate different translations for the input sentence by linearly interpolating it with different sentence pairs sampled from the training corpus when decoding. To further improve the faithfulness and diversity of the translations, we propose two simple but effective approaches to select diverse sentence pairs in the training corpus and adjust the interpolation weight for each pair correspondingly. Moreover, by controlling the interpolation weight, our method can achieve the trade-off between faithfulness and diversity without any additional training, which is required in most of the previous methods. Experiments on WMT'16 en-ro, WMT'14 en-de, and WMT'17 zh-en are conducted to show that our method substantially outperforms all previous diverse machine translation methods.