# Can Neural Machine Translation be Improved with User Feedback?

arXiv: https://arxiv.org/abs/1804.05958 · HTML: https://mlsanity.com/paper/1804.05958/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2018.

## Built on by (most cited first, 30 of 72)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [Aligning Text-to-Image Models using Human Feedback](https://mlsanity.com/paper/2302.12192/) (2023)
- [Humans or LLMs as the Judge? A Study on Judgement Biases](https://mlsanity.com/paper/2402.10669/) (2024)
- [Recursively Summarizing Books with Human Feedback](https://mlsanity.com/paper/2109.10862/) (2021)
- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [RAIN: Your Language Models Can Align Themselves without Finetuning](https://mlsanity.com/paper/2309.07124/) (2023)
- [Chain of Hindsight Aligns Language Models with Feedback](https://mlsanity.com/paper/2302.02676/) (2023)
- [DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback](https://mlsanity.com/paper/2311.10081/) (2023)
- [RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs](https://mlsanity.com/paper/2404.08555/) (2024)
- [Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning](https://mlsanity.com/paper/1805.10627/) (2018)
- [Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation](https://mlsanity.com/paper/2305.00955/) (2023)
- [Learning to Generate Better Than Your LLM](https://mlsanity.com/paper/2306.11816/) (2023)
- [Everyone Deserves A Reward: Learning Customized Human Preferences](https://mlsanity.com/paper/2309.03126/) (2023)
- [Putting Humans in the Natural Language Processing Loop: A Survey](https://mlsanity.com/paper/2103.04044/) (2021)
- [Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences](https://mlsanity.com/paper/2403.07230/) (2024)
- [M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions](https://mlsanity.com/paper/2405.16420/) (2024)
- [Tuning computer vision models with task rewards](https://mlsanity.com/paper/2302.08242/) (2023)
- [Improving Machine Translation with Human Feedback: An Exploration of Quality Estimation as a Reward Model](https://mlsanity.com/paper/2401.12873/) (2024)
- [EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization](https://mlsanity.com/paper/2405.15189/) (2024)
- [Aligning Language Models with Human Preferences via a Bayesian Approach](https://mlsanity.com/paper/2310.05782/) (2023)
- [Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game](https://mlsanity.com/paper/2311.08045/) (2023)
- [Reliability Testing for Natural Language Processing Systems](https://mlsanity.com/paper/2105.02590/) (2021)
- [HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent](https://mlsanity.com/paper/2406.00215/) (2024)
- [Continual Learning for Instruction Following from Realtime Feedback](https://mlsanity.com/paper/2212.09710/) (2022)
- [Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior](https://mlsanity.com/paper/2108.04812/) (2021)
- [Sample Efficient Preference Alignment in LLMs via Active Exploration](https://mlsanity.com/paper/2312.00267/) (2023)
- [Privately Aligning Language Models with Reinforcement Learning](https://mlsanity.com/paper/2310.16960/) (2023)

## Abstract (arXiv)

We present the first real-world application of methods for improving neural machine translation (NMT) with human reinforcement, based on explicit and implicit user feedback collected on the eBay e-commerce platform. Previous work has been confined to simulation experiments, whereas in this paper we work with real logged feedback for offline bandit learning of NMT parameters. We conduct a thorough analysis of the available explicit user judgments---five-star ratings of translation quality---and show that they are not reliable enough to yield significant improvements in bandit learning. In contrast, we successfully utilize implicit task-based feedback collected in a cross-lingual search task to improve task-specific and machine translation quality metrics.