# Summary Level Training of Sentence Rewriting for Abstractive Summarization

arXiv: https://arxiv.org/abs/1909.08752 · HTML: https://mlsanity.com/paper/1909.08752/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2019.

## Built on by (most cited first, 23 of 23)

- [Extractive Summarization via ChatGPT for Faithful Summary Generation](https://mlsanity.com/paper/2304.04193/) (2023)
- [A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models](https://mlsanity.com/paper/2406.11289/) (2024)
- [A Survey on Long Text Modeling with Transformers](https://mlsanity.com/paper/2302.14502/) (2023)
- [What's in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarization](https://mlsanity.com/paper/2105.00816/) (2021)
- [DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization](https://mlsanity.com/paper/2110.08168/) (2021)
- [DiffuSum: Generation Enhanced Extractive Summarization with Diffusion](https://mlsanity.com/paper/2305.01735/) (2023)
- [Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization](https://mlsanity.com/paper/2204.13761/) (2022)
- [Contextualized Rewriting for Text Summarization](https://mlsanity.com/paper/2102.00385/) (2021)
- [SgSum: Transforming Multi-document Summarization into Sub-graph Selection](https://mlsanity.com/paper/2110.12645/) (2021)
- [COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization](https://mlsanity.com/paper/2209.14569/) (2022)
- [RewardsOfSum: Exploring Reinforcement Learning Rewards for Summarisation](https://mlsanity.com/paper/2106.04080/) (2021)
- [A General Contextualized Rewriting Framework for Text Summarization](https://mlsanity.com/paper/2207.05948/) (2022)
- [Universal Evasion Attacks on Summarization Scoring](https://mlsanity.com/paper/2210.14260/) (2022)
- [Benchmarking LLMs on the Semantic Overlap Summarization Task](https://mlsanity.com/paper/2402.17008/) (2024)
- [Transductive Learning for Abstractive News Summarization](https://mlsanity.com/paper/2104.09500/) (2021)
- [Reinforcement Learning without Human Feedback for Last Mile Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2408.16753/) (2024)
- [Centrality Meets Centroid: A Graph-based Approach for Unsupervised Document Summarization](https://mlsanity.com/paper/2103.15327/) (2021)
- [Reinforced Abstractive Summarization with Adaptive Length Controlling](https://mlsanity.com/paper/2112.07534/) (2021)
- [Multi-Narrative Semantic Overlap Task: Evaluation and Benchmark](https://mlsanity.com/paper/2201.05294/) (2022)
- [GEMINI: Controlling the Sentence-level Writing Style for Abstractive Text Summarization](https://mlsanity.com/paper/2304.03548/) (2023)
- [A Hybrid Strategy for Chat Transcript Summarization](https://mlsanity.com/paper/2402.01510/) (2024)
- [MARS: Multilingual Aspect-centric Review Summarisation](https://mlsanity.com/paper/2410.09991/) (2024)
- [OrderSum: Semantic Sentence Ordering for Extractive Summarization](https://mlsanity.com/paper/2502.16180/) (2025)

## Abstract (arXiv)

As an attempt to combine extractive and abstractive summarization, Sentence Rewriting models adopt the strategy of extracting salient sentences from a document first and then paraphrasing the selected ones to generate a summary. However, the existing models in this framework mostly rely on sentence-level rewards or suboptimal labels, causing a mismatch between a training objective and evaluation metric. In this paper, we present a novel training signal that directly maximizes summary-level ROUGE scores through reinforcement learning. In addition, we incorporate BERT into our model, making good use of its ability on natural language understanding. In extensive experiments, we show that a combination of our proposed model and training procedure obtains new state-of-the-art performance on both CNN/Daily Mail and New York Times datasets. We also demonstrate that it generalizes better on DUC-2002 test set.