# Rethinking Translation Memory Augmented Neural Machine Translation

arXiv: https://arxiv.org/abs/2306.06948 · HTML: https://mlsanity.com/paper/2306.06948/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 52 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2010.00710/) (2020)
- [Unsupervised Domain Clusters in Pretrained Language Models](https://mlsanity.com/paper/2004.02105/) (2020)
- [Rethinking Bias-Variance Trade-off for Generalization of Neural Networks](https://mlsanity.com/paper/2002.11328/) (2020)
- [Guiding Neural Machine Translation with Retrieved Translation Pieces](https://mlsanity.com/paper/1804.02559/) (2018)
- [Neural Machine Translation with Monolingual Translation Memory](https://mlsanity.com/paper/2105.11269/) (2021)
- [Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data](https://mlsanity.com/paper/2203.08773/) (2022)
- [Adaptive Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2105.13022/) (2021)
- [Fast Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2105.14528/) (2021)
- [Efficient Cluster-Based k-Nearest-Neighbor Machine Translation](https://mlsanity.com/paper/2204.06175/) (2022)
- [Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation](https://mlsanity.com/paper/2109.06604/) (2021)
- [TranSmart: A Practical Interactive Machine Translation System](https://mlsanity.com/paper/2105.13072/) (2021)
- [Memory-augmented Neural Machine Translation](https://mlsanity.com/paper/1708.02005/) (2017)
- [Non-Parametric Domain Adaptation for End-to-End Speech Translation](https://mlsanity.com/paper/2205.11211/) (2022)
- [Neural Machine Translation with Contrastive Translation Memories](https://mlsanity.com/paper/2212.03140/) (2022)
- [Simple and Scalable Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2302.12188/) (2023)
- [Faster Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2112.08152/) (2021)
- [Improving Retrieval Augmented Neural Machine Translation by Controlling Source and Fuzzy-Match Interactions](https://mlsanity.com/paper/2210.05047/) (2022)
- [What Knowledge Is Needed? Towards Explainable Memory for kNN-MT Domain Adaptation](https://mlsanity.com/paper/2211.04052/) (2022)
- [Learning to Reuse Translations: Guiding Neural Machine Translation with Examples](https://mlsanity.com/paper/1911.10732/) (2019)
- [Federated Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2302.12211/) (2023)
- [Learning Decoupled Retrieval Representation for Nearest Neighbour Neural Machine Translation](https://mlsanity.com/paper/2209.08738/) (2022)
- [Improving Robustness of Retrieval Augmented Translation via Shuffling of Suggestions](https://mlsanity.com/paper/2210.05059/) (2022)

## Built on by (most cited first, 1 of 1)

- [IMTLab: An Open-Source Platform for Building, Evaluating, and Diagnosing Interactive Machine Translation Systems](https://mlsanity.com/paper/2310.11163/) (2023)

## Abstract (arXiv)

This paper rethinks translation memory augmented neural machine translation (TM-augmented NMT) from two perspectives, i.e., a probabilistic view of retrieval and the variance-bias decomposition principle. The finding demonstrates that TM-augmented NMT is good at the ability of fitting data (i.e., lower bias) but is more sensitive to the fluctuations in the training data (i.e., higher variance), which provides an explanation to a recently reported contradictory phenomenon on the same translation task: TM-augmented NMT substantially advances vanilla NMT under the high-resource scenario whereas it fails under the low-resource scenario. Then we propose a simple yet effective TM-augmented NMT model to promote the variance and address the contradictory phenomenon. Extensive experiments show that the proposed TM-augmented NMT achieves consistent gains over both conventional NMT and existing TM-augmented NMT under two variance-preferable (low-resource and plug-and-play) scenarios as well as the high-resource scenario.