# Efficient k-Nearest-Neighbor Machine Translation with Dynamic Retrieval

arXiv: https://arxiv.org/abs/2406.06073 · HTML: https://mlsanity.com/paper/2406.06073/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 10 Jun 2024.

## Builds on (most cited first, 20 of 20)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2010.00710/) (2020)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [Unsupervised Domain Clusters in Pretrained Language Models](https://mlsanity.com/paper/2004.02105/) (2020)
- [Efficient Nearest Neighbor Language Models](https://mlsanity.com/paper/2109.04212/) (2021)
- [Norm-Based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/2006.02014/) (2020)
- [Adaptive Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2105.13022/) (2021)
- [Efficient Cluster-Based k-Nearest-Neighbor Machine Translation](https://mlsanity.com/paper/2204.06175/) (2022)
- [Chunk-based Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2205.12230/) (2022)
- [Towards Robust k-Nearest-Neighbor Machine Translation](https://mlsanity.com/paper/2210.08808/) (2022)
- [Simple and Scalable Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2302.12188/) (2023)
- [What Knowledge Is Needed? Towards Explainable Memory for kNN-MT Domain Adaptation](https://mlsanity.com/paper/2211.04052/) (2022)
- [INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2306.06381/) (2023)
- [Multilingual k-Nearest-Neighbor Machine Translation](https://mlsanity.com/paper/2310.14644/) (2023)
- [Towards Faster k-Nearest-Neighbor Machine Translation](https://mlsanity.com/paper/2312.07419/) (2023)

## Built on by (most cited first, 2 of 2)

- [Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.12460/) (2025)
- [Angular Dispersion Accelerates k-Nearest Neighbors Machine Translation](https://mlsanity.com/paper/2509.16729/) (2025)

## Abstract (arXiv)

To achieve non-parametric NMT domain adaptation, k-Nearest-Neighbor Machine Translation (kNN-MT) constructs an external datastore to store domain-specific translation knowledge, which derives a kNN distribution to interpolate the prediction distribution of the NMT model via a linear interpolation coefficient λ. Despite its success, kNN retrieval at each timestep leads to substantial time overhead. To address this issue, dominant studies resort to kNN-MT with adaptive retrieval (kNN-MT-AR), which dynamically estimates λ and skips kNN retrieval if λ is less than a fixed threshold. Unfortunately, kNN-MT-AR does not yield satisfactory results. In this paper, we first conduct a preliminary study to reveal two key limitations of kNN-MT-AR: 1) the optimization gap leads to inaccurate estimation of λ for determining kNN retrieval skipping, and 2) using a fixed threshold fails to accommodate the dynamic demands for kNN retrieval at different timesteps. To mitigate these limitations, we then propose kNN-MT with dynamic retrieval (kNN-MT-DR) that significantly extends vanilla kNN-MT in two aspects. Firstly, we equip kNN-MT with a MLP-based classifier for determining whether to skip kNN retrieval at each timestep. Particularly, we explore several carefully-designed scalar features to fully exert the potential of the classifier. Secondly, we propose a timestep-aware threshold adjustment method to dynamically generate the threshold, which further improves the efficiency of our model. Experimental results on the widely-used datasets demonstrate the effectiveness and generality of our model.footnoteOur code is available at urlhttps://github.com/DeepLearnXMU/knn-mt-dr.