# Simply Trainable Nearest Neighbour Machine Translation with GPU Inference

arXiv: https://arxiv.org/abs/2407.19965 · HTML: https://mlsanity.com/paper/2407.19965/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 13 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2010.00710/) (2020)
- [Scalable and Efficient MoE Training for Multitask Multilingual Models](https://mlsanity.com/paper/2109.10465/) (2021)
- [Adaptive Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2105.13022/) (2021)
- [Fast Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2105.14528/) (2021)
- [Learning Kernel-Smoothed Machine Translation with Retrieved Examples](https://mlsanity.com/paper/2109.09991/) (2021)
- [Efficient Cluster-Based k-Nearest-Neighbor Machine Translation](https://mlsanity.com/paper/2204.06175/) (2022)
- [Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation](https://mlsanity.com/paper/2109.06604/) (2021)
- [Towards Robust k-Nearest-Neighbor Machine Translation](https://mlsanity.com/paper/2210.08808/) (2022)
- [Simple and Scalable Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2302.12188/) (2023)

## Abstract (arXiv)

Nearest neighbor machine translation is a successful approach for fast domain adaption, which interpolates the pre-trained transformers with domain-specific token-level k-nearest-neighbor (kNN) retrieval without retraining. Despite kNN MT's success, searching large reference corpus and fixed interpolation between the kNN and pre-trained model led to computational complexity and translation quality challenges. Among other papers, Dai et al. proposed methods to obtain a small number of reference samples dynamically for which they introduced a distance-aware interpolation method using an equation that includes free parameters. This paper proposes a simply trainable nearest neighbor machine translation and carry out inference experiments on GPU. Similar to Dai et al., we first adaptively construct a small datastore for each input sentence. Second, we train a single-layer network for the interpolation coefficient between the knnMT and pre-trained result to automatically interpolate in different domains. Experimental results on different domains show that our proposed method either improves or sometimes maintain the translation quality of methods in Dai et al. while being automatic. In addition, our GPU inference results demonstrate that knnMT can be integrated into GPUs with a drop of only 5% in terms of speed.