# Improving Retrieval Augmented Neural Machine Translation by Controlling Source and Fuzzy-Match Interactions

arXiv: https://arxiv.org/abs/2210.05047 · HTML: https://mlsanity.com/paper/2210.05047/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2010.00710/) (2020)
- [Unsupervised Domain Clusters in Pretrained Language Models](https://mlsanity.com/paper/2004.02105/) (2020)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [Guiding Neural Machine Translation with Retrieved Translation Pieces](https://mlsanity.com/paper/1804.02559/) (2018)
- [Neural Machine Translation with Monolingual Translation Memory](https://mlsanity.com/paper/2105.11269/) (2021)
- [Adaptive Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2105.13022/) (2021)
- [compare-mt: A Tool for Holistic Comparison of Language Generation Systems](https://mlsanity.com/paper/1903.07926/) (2019)
- [Learning Hidden Unit Contributions for Unsupervised Acoustic Model Adaptation](https://mlsanity.com/paper/1601.02828/) (2016)
- [Fast Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2105.14528/) (2021)
- [Fast Domain Adaptation for Neural Machine Translation](https://mlsanity.com/paper/1612.06897/) (2016)
- [The Sockeye 2 Neural Machine Translation Toolkit at AMTA 2020](https://mlsanity.com/paper/2008.04885/) (2020)
- [Search Engine Guided Non-Parametric Neural Machine Translation](https://mlsanity.com/paper/1705.07267/) (2017)

## Built on by (most cited first, 7 of 7)

- [Understanding Retrieval Robustness for Retrieval-Augmented Image Captioning](https://mlsanity.com/paper/2406.02265/) (2024)
- [Retrieval-Augmented Machine Translation with Unstructured Knowledge](https://mlsanity.com/paper/2412.04342/) (2024)
- [Survey on Memory-Augmented Neural Networks: Cognitive Insights to AI Applications](https://mlsanity.com/paper/2312.06141/) (2023)
- [Prompting Large Language Models with Human Error Markings for Self-Correcting Machine Translation](https://mlsanity.com/paper/2406.02267/) (2024)
- [Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach](https://mlsanity.com/paper/2409.09009/) (2024)
- [Towards Example-Based NMT with Multi-Levenshtein Transformers](https://mlsanity.com/paper/2310.08967/) (2023)
- [Rethinking Translation Memory Augmented Neural Machine Translation](https://mlsanity.com/paper/2306.06948/) (2023)

## Abstract (arXiv)

We explore zero-shot adaptation, where a general-domain model has access to customer or domain specific parallel data at inference time, but not during training. We build on the idea of Retrieval Augmented Translation (RAT) where top-k in-domain fuzzy matches are found for the source sentence, and target-language translations of those fuzzy-matched sentences are provided to the translation model at inference time. We propose a novel architecture to control interactions between a source sentence and the top-k fuzzy target-language matches, and compare it to architectures from prior work. We conduct experiments in two language pairs (En-De and En-Fr) by training models on WMT data and testing them with five and seven multi-domain datasets, respectively. Our approach consistently outperforms the alternative architectures, improving BLEU across language pair, domain, and number k of fuzzy matches.