mlsanityData to 14 Oct 2026

Sequence Level Training with Recurrent Neural Networks

arXiv 1511.06732 ↗First posted 20 Nov 2015cs.LG cs.CLcited by 576

Not on the map. Why?

Cited by 48 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Many natural language processing applications use language models to generate text. These models are typically trained to predict the next word in a sequence, given the previous words and some context such as an image. However, at test time the model is expected to generate the entire sequence from scratch. This discrepancy makes generation brittle, as errors may accumulate along the way. We address this issue by proposing a novel sequence level training algorithm that directly optimizes the metric used at test time, such as BLEU or ROUGE. On three different tasks, our approach outperforms several strong baselines for greedy generation. The method is also competitive when these baselines employ beam search, while being several times faster.

Key facts. Cited by 576 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2017: 2; 2018: 0; 2019: 4; 2020: 6; 2021: 98; 2022: 101; 2023: 98; 2024: 76; 2025: 59; 2026: 132.

576 papers.

Sort:

By area:

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2020Learning to summarize from human feedbackcited by 2,588
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  5. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  6. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  8. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  9. 2023Steering Language Models With Activation Engineeringcited by 922
  10. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  11. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  12. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
38 more
  1. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  2. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
  3. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  4. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 268
  5. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  6. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 243
  7. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 237
  8. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 228
  9. 2022Unified Structure Generation for Universal Information Extractioncited by 221
  10. 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
  11. 2023ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Modelscited by 187
  12. 2026Entropy-Aware On-Policy Distillation of Language Modelscited by 175
  13. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  14. 2024The pitfalls of next-token predictioncited by 160
  15. 2024Stream of Search (SoS): Learning to Search in Languagecited by 145
  16. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 138
  17. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 138
  18. 2022Offline RL for Natural Language Generation with Implicit Language Q Learningcited by 138
  19. 2023Structured Chain-of-Thought Prompting for Code Generationcited by 138
  20. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 135
  21. 2021SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarizationcited by 132
  22. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  23. 2022Calibrating Sequence likelihood Improves Conditional Language Generationcited by 121
  24. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 120
  25. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  26. 2022Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generationcited by 108
  27. 2024Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMscited by 108
  28. 2023GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretationcited by 103
  29. 2023Elucidating the Exposure Bias in Diffusion Modelscited by 101
  30. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 99
  31. 2021Survey of Low-Resource Machine Translationcited by 92
  32. 2023Input Perturbation Reduces Exposure Bias in Diffusion Modelscited by 90
  33. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 90
  34. 2023Monocular Depth Estimation using Diffusion Modelscited by 87
  35. 2025SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RLcited by 86
  36. 2021Computer-Aided Design as Languagecited by 82
  37. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 82
  38. 2023On Learning to Summarize with Large Language Models as Referencescited by 80

50 most cited of 576 All 576 (JSON)