mlsanityData to 7 Oct 2026

Bandit Structured Prediction for Neural Sequence-to-Sequence Learning

arXiv 1704.06497 ↗First posted 21 Apr 2017stat.ML cs.CL cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

Bandit structured prediction describes a stochastic optimization framework where learning is performed from partial feedback. This feedback is received in the form of a task loss evaluation to a predicted output structure, without having access to gold standard structures. We advance this framework by lifting linear bandit learning to neural sequence-to-sequence learning problems using attention-based recurrent neural networks. Furthermore, we show how to incorporate control variates into our learning algorithms for variance reduction and improved generalization. We present an evaluation on a neural machine translation task that shows improvements of up to 5.89 BLEU points for domain adaptation from simulated bandit feedback.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 1; 2018: 1; 2019: 0; 2020: 0; 2021: 6; 2022: 3; 2023: 5; 2024: 4; 2025: 5; 2026: 6.

31 papers.

Sort:

By area:

  1. 2023ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Modelscited by 184
  2. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  3. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  4. 2017Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedbackcited by 71
  5. 2018Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learningcited by 57
  6. 2025DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPOcited by 57
  7. 2023The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Valuescited by 53
  8. 2024Improving Machine Translation with Human Feedback: An Exploration of Quality Estimation as a Reward Modelcited by 33
  9. 2021Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 32
  10. 2021Interactive Learning from Activity Descriptioncited by 26
  11. 2021Continual Learning for Grounded Instruction Generation by Observing Human Following Behaviorcited by 19
  12. 2021Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Banditscited by 16
19 more
  1. 2023ESRL: Efficient Sampling-based Reinforcement Learning for Sequence Generationcited by 15
  2. 2022Teacher Forcing Recovers Reward Functions for Text Generationcited by 11
  3. 2024Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappingscited by 11
  4. 2025Aligning to What? Limits to RLHF Based Alignmentcited by 8
  5. 2021Stochastic Multi-Armed Bandits with Control Variatescited by 7
  6. 2022MAD for Robust Reinforcement Learning in Machine Translationcited by 7
  7. 2023Exploiting Correlated Auxiliary Feedback in Parameterized Banditscited by 6
  8. 2024LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectivescited by 6
  9. 2024SumRecom: A Personalized Summarization Approach by Learning from Users' Feedbackcited by 4
  10. 2025Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Securitycited by 4
  11. 2021Fixing exposure bias with imitation learning needs powerful oraclescited by 3
  12. 2023SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspherescited by 2
  13. 2025One Framework to Rule Them All: Unifying RL-Based and RL-Free Methods in RLHFcited by 2
  14. 2026Decision-Focused On-Policy Learning for Contextual Linear Optimization with Partial Feedbackcited by 1
  15. 2026Rate or Fate? RLV^varepsilonR: Reinforcement Learning with Verifiable Noisy Rewardscited by 0
  16. 2026Stochastic Multi-Armed Bandits with Limited Control Variatescited by 0
  17. 2026Reference-Free Reinforcement Learning Fine-Tuning for MT: A Seq2Seq Perspectivecited by 0
  18. 2026Bridging the Usability Gap: Lessons from Interpreting Studies for Machine Interpreting Designcited by 0
  19. 2026MICRO: Multi-Fidelity Active Search for Severe Error Discoverycited by 0