mlsanityData to 7 Oct 2026

Non-Parametric Online Learning from Human Feedback for Neural Machine Translation

arXiv 2109.11136 ↗First posted 23 Sep 2021cs.CLcited by 8

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

We study the problem of online learning with human feedback in the human-in-the-loop machine translation, in which the human translators revise the machine-generated translations and then the corrected translations are used to improve the neural machine translation (NMT) system. However, previous methods require online model updating or additional translation memory networks to achieve high-quality performance, making them inflexible and inefficient in practice. In this paper, we propose a novel non-parametric online learning method without changing the model structure. This approach introduces two k-nearest-neighbor (knn) modules: one module memorizes the human feedback, which is the correct sentences provided by human translators, while the other balances the usage of the history human feedback and original NMT models adaptively. Experiments conducted on EMEA and JRC-Acquis benchmarks demonstrate that our proposed method obtains substantial improvements on translation accuracy and achieves better adaptation performance with less repeating human correction operations.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 25 of its 42 references that are in the map. First posted on arXiv 23 Sep 2021.

Builds on

25 of 42 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2017Billion-scale similarity search with GPUscited by 2,477
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  6. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  7. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  8. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
  9. 2020Nearest Neighbor Machine Translationcited by 209
  10. 2019Facebook FAIR's WMT19 News Translation Task Submissioncited by 179
  11. 2018Achieving Human Parity on Automatic Chinese to English News Translationcited by 119
  12. 2018Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluationcited by 88
13 more
  1. 2018Guiding Neural Machine Translation with Retrieved Translation Piecescited by 63
  2. 2021Neural Machine Translation with Monolingual Translation Memorycited by 61
  3. 2021Adaptive Nearest Neighbor Machine Translationcited by 58
  4. 2017Learning to Remember Translation History with a Continuous Cachecited by 39
  5. 2019Non-Parametric Adaptation for Neural Machine Translationcited by 34
  6. 2018Joint Training for Neural Machine Translation Models with Monolingual Datacited by 20
  7. 2021Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translationcited by 20
  8. 2018Online Learning for Effort Reduction in Interactive Neural Machine Translationcited by 9
  9. 2019Correct-and-Memorize: Learning to Translate from Interactive Revisionscited by 6
  10. 2017A User-Study on Online Adaptation of Neural Machine Translation to Human Post-Editscited by 5
  11. 2019Learning to Reuse Translations: Guiding Neural Machine Translation with Examplescited by 5
  12. 2019Incremental Adaptation of NMT for Professional Post-editors: A User Studycited by 3
  13. 2019Demonstration of a Neural Machine Translation System with Online Learning for Translatorscited by 2

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 5; 2024: 1.

8 papers.

  1. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  2. 2023The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Valuescited by 53
  3. 2022Improving Few-Shot Performance of Language Models via Nearest Neighbor Calibrationcited by 24
  4. 2022Non-Parametric Domain Adaptation for End-to-End Speech Translationcited by 12
  5. 2023Simple and Scalable Nearest Neighbor Machine Translationcited by 12
  6. 2024ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotationcited by 6
  7. 2023Federated Nearest Neighbor Machine Translationcited by 5
  8. 2023Human-in-the-loop Machine Translation with Large Language Modelcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025Efficient Continual Learning in Neural Machine Translation: A Low-Rank Adaptation Approach
  2. 2025Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation
  3. 2023Aligning Neural Machine Translation Models: Human Feedback in Training and Inference
  4. 2023INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation
  5. 2024Improving Machine Translation with Human Feedback: An Exploration of Quality Estimation as a Reward Model
  6. 2024Multi-perspective Alignment for Increasing Naturalness in Neural Machine Translation
  7. 2017Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback
  8. 2026North Small Translate: Advanced Cost-Effective Translation (Cohere CAT+)