mlsanityData to 14 Oct 2026

Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation

arXiv 2106.08942 ↗First posted 16 Jun 2021cs.CLcited by 33

Not on the map. Why?

Abstract (arXiv)

Policy gradient algorithms have found wide adoption in NLP, but have recently become subject to criticism, doubting their suitability for NMT. Choshen et al. (2020) identify multiple weaknesses and suspect that their success is determined by the shape of output distributions rather than the reward. In this paper, we revisit these claims and study them under a wider range of configurations. Our experiments on in-domain and cross-domain adaptation reveal the importance of exploration and reward scaling, and provide empirical counter-evidence to these claims.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 16 Jun 2021.

Builds on

22 of 29 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  4. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
  5. 2017Six Challenges for Neural Machine Translationcited by 335
  6. 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
  7. 2016An Actor-Critic Algorithm for Sequence Predictioncited by 159
  8. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  9. 2015Minimum Risk Training for Neural Machine Translationcited by 115
  10. 2019Beyond BLEU: Training Neural Machine Translation with Semantic Similaritycited by 84
  11. 2019On the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 82
  12. 2018A Study of Reinforcement Learning for Neural Machine Translationcited by 74
10 more
  1. 2017Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedbackcited by 72
  2. 2017Paraphrase Generation with Deep Reinforcement Learningcited by 58
  3. 2017Classical Structured Prediction Losses for Sequence to Sequence Learningcited by 55
  4. 2019Joey NMT: A Minimalist NMT Toolkit for Novicescited by 39
  5. 2017Bandit Structured Prediction for Neural Sequence-to-Sequence Learningcited by 31
  6. 2018Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translationcited by 29
  7. 2017Learning to Skim Textcited by 18
  8. 2016Stochastic Structured Prediction under Bandit Feedbackcited by 12
  9. 2019Deep Reinforcement Learning For Modeling Chit-Chat Dialog With Discrete Attributescited by 11
  10. 2017A Shared Task on Bandit Learning for Machine Translationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 10; 2023: 13; 2024: 5; 2025: 3; 2026: 1.

33 papers.

Sort:

By area:

  1. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  2. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  3. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 90
  4. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 88
  5. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 82
  6. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  7. 2023Learning to Generate Better Than Your LLMcited by 49
  8. 2025MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learningcited by 48
  9. 2023Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuningcited by 33
  10. 2022Reward Gaming in Conditional Text Generationcited by 30
  11. 2023Vanishing Gradients in Reinforcement Finetuning of Language Modelscited by 27
  12. 2021Boosting Search Engines with Interactive Agentscited by 19
21 more
  1. 2023Aligning Neural Machine Translation Models: Human Feedback in Training and Inferencecited by 19
  2. 2025BLEUBERI: BLEU is a surprisingly effective reward for instruction followingcited by 17
  3. 2023ESRL: Efficient Sampling-based Reinforcement Learning for Sequence Generationcited by 15
  4. 2023Prompt-Based Length Controlled Generation with Reinforcement Learningcited by 14
  5. 2023Reinforcement Learning for Generative AI: A Surveycited by 11
  6. 2024Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappingscited by 11
  7. 2023Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challengescited by 10
  8. 2022MAD for Robust Reinforcement Learning in Machine Translationcited by 7
  9. 2022Multi-Granularity Optimization for Non-Autoregressive Translationcited by 6
  10. 2023On the Effectiveness of Offline RL for Dialogue Response Generationcited by 6
  11. 2023Learning Evaluation Models from Large Language Models for Sequence Generationcited by 6
  12. 2022Hierarchical Context Tagging for Utterance Rewritingcited by 4
  13. 2022Joint Generator-Ranker Learning for Natural Language Generationcited by 4
  14. 2022Switching to Discriminative Image Captioning by Relieving a Bottleneck of Reinforcement Learningcited by 4
  15. 2024Style Transfer with Multi-iteration Preference Optimizationcited by 4
  16. 2025RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translationcited by 4
  17. 2023Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compressioncited by 3
  18. 2022Policy Gradient Algorithms with Monte Carlo Tree Learning for Non-Markov Decision Processescited by 2
  19. 2024Reinforcement Learning for Edit-Based Non-Autoregressive Neural Machine Translationcited by 1
  20. 2024Optimizing the Training Schedule of Multilingual NMT using Reinforcement Learningcited by 0
  21. 2026Metric-Driven Reinforcement Learning for Object Hallucination Reduction in LVLM-Based Image Captioningcited by 0