mlsanityData to 14 Oct 2026

A Deep Reinforced Model for Abstractive Summarization

arXiv 1705.04304 ↗First posted 11 May 2017cs.CLcited by 365

Not on the map. Why?

Cited by 35 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Attentional, RNN-based encoder-decoder models for abstractive summarization have achieved good performance on short input and output sequences. For longer documents and summaries however these models often include repetitive and incoherent phrases. We introduce a neural network model with a novel intra-attention that attends over the input and continuously generated output separately, and a new training method that combines standard supervised word prediction and reinforcement learning (RL). Models trained only with supervised learning often exhibit "exposure bias" - they assume ground truth is provided at each step during training. However, when standard word prediction is combined with the global sequence prediction training of RL the resulting summaries become more readable. We evaluate this model on the CNN/Daily Mail and New York Times datasets. Our model obtains a 41.16 ROUGE-1 score on the CNN/Daily Mail dataset, an improvement over previous state-of-the-art models. Human evaluation also shows that our model produces higher quality summaries.

Key facts. Cited by 365 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2017: 1; 2018: 1; 2019: 1; 2020: 3; 2021: 109; 2022: 89; 2023: 72; 2024: 53; 2025: 21; 2026: 15.

365 papers.

Sort:

By area:

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2020Learning to summarize from human feedbackcited by 2,588
  3. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  4. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  5. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  6. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  7. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  8. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
  9. 2023SLiC-HF: Sequence Likelihood Calibration with Human Feedbackcited by 401
  10. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  11. 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 328
  12. 2022Masked Autoencoders that Listencited by 273
38 more
  1. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  2. 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 244
  3. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 237
  4. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 228
  5. 2021BookSum: A Collection of Datasets for Long-form Narrative Summarizationcited by 184
  6. 2021DialogSum: A Real-Life Scenario Dialogue Summarization Datasetcited by 183
  7. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  8. 2024The pitfalls of next-token predictioncited by 160
  9. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 155
  10. 2022Offline RL for Natural Language Generation with Implicit Language Q Learningcited by 138
  11. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 135
  12. 2021SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarizationcited by 132
  13. 2024Language Models Learn to Mislead Humans via RLHFcited by 124
  14. 2022Pretrained Language Models for Text Generation: A Surveycited by 122
  15. 2022Calibrating Sequence likelihood Improves Conditional Language Generationcited by 121
  16. 2021Screen2Words: Automatic Mobile UI Summarization with Multimodal Learningcited by 120
  17. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 120
  18. 2023f-Divergence Minimization for Sequence-Level Knowledge Distillationcited by 114
  19. 2022Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Languagecited by 110
  20. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 95
  21. 2022Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generationcited by 94
  22. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 88
  23. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 87
  24. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 82
  25. 2023On Learning to Summarize with Large Language Models as Referencescited by 80
  26. 2021SummScreen: A Dataset for Abstractive Screenplay Summarizationcited by 79
  27. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 78
  28. 2022An Empirical Survey on Long Document Summarization: Datasets, Models and Metricscited by 74
  29. 2023Transformers as Support Vector Machinescited by 74
  30. 2021DOC2PPT: Automatic Presentation Slides Generation from Scientific Documentscited by 70
  31. 2021CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learningcited by 68
  32. 2023Align and Attend: Multimodal Summarization with Dual Contrastive Lossescited by 65
  33. 2021Can We Automate Scientific Reviewing?cited by 63
  34. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  35. 2022MAViL: Masked Audio-Video Learnerscited by 59
  36. 2023Max-Margin Token Selection in Attention Mechanismcited by 58
  37. 2018Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learningcited by 57
  38. 2023Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generationcited by 55

50 most cited of 365 All 365 (JSON)