# Improving Abstraction in Text Summarization

arXiv: https://arxiv.org/abs/1808.07913 · HTML: https://mlsanity.com/paper/1808.07913/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2018.

## Built on by (most cited first, 30 of 35)

- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors](https://mlsanity.com/paper/2205.12854/) (2022)
- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2105.10311/) (2021)
- [Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization](https://mlsanity.com/paper/2305.13091/) (2023)
- [Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization](https://mlsanity.com/paper/2108.13684/) (2021)
- [MReD: A Meta-Review Dataset for Structure-Controllable Text Generation](https://mlsanity.com/paper/2110.07474/) (2021)
- [Hierarchical Learning for Generation with Long Source Sequences](https://mlsanity.com/paper/2104.07545/) (2021)
- [Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand](https://mlsanity.com/paper/2112.04139/) (2021)
- [Spurious Correlations in Reference-Free Evaluation of Text Generation](https://mlsanity.com/paper/2204.09890/) (2022)
- [Automatic Construction of Evaluation Suites for Natural Language Generation Datasets](https://mlsanity.com/paper/2106.09069/) (2021)
- [Towards Better Evaluation of Instruction-Following: A Case-Study in Summarization](https://mlsanity.com/paper/2310.08394/) (2023)
- [Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Survey](https://mlsanity.com/paper/2311.09212/) (2023)
- [Controllable Summarization with Constrained Markov Decision Process](https://mlsanity.com/paper/2108.03405/) (2021)
- [Automatic Text Summarization Methods: A Comprehensive Review](https://mlsanity.com/paper/2204.01849/) (2022)
- [Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization](https://mlsanity.com/paper/2305.16784/) (2023)
- [Abstractive Text Summarization: State of the Art, Challenges, and Improvements](https://mlsanity.com/paper/2409.02413/) (2024)
- [Evaluating the Tradeoff Between Abstractiveness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2108.02859/) (2021)
- [NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklist](https://mlsanity.com/paper/2305.08566/) (2023)
- [One Prompt To Rule Them All: LLMs for Opinion Summary Evaluation](https://mlsanity.com/paper/2402.11683/) (2024)
- [ALLURE: Auditing and Improving LLM-based Evaluation of Text using Iterative In-Context-Learning](https://mlsanity.com/paper/2309.13701/) (2023)
- [METAL: Towards Multilingual Meta-Evaluation](https://mlsanity.com/paper/2404.01667/) (2024)
- [Multi-Dimensional Optimization for Text Summarization via Reinforcement Learning](https://mlsanity.com/paper/2406.00303/) (2024)
- [Summary Explorer: Visualizing the State of the Art in Text Summarization](https://mlsanity.com/paper/2108.01879/) (2021)
- [SumRecom: A Personalized Summarization Approach by Learning from Users' Feedback](https://mlsanity.com/paper/2408.07294/) (2024)
- [May the Force Be with Your Copy Mechanism: Enhanced Supervised-Copy Method for Natural Language Generation](https://mlsanity.com/paper/2112.10360/) (2021)
- [Universal Evasion Attacks on Summarization Scoring](https://mlsanity.com/paper/2210.14260/) (2022)
- [Evaluating the Evaluator: Summarization Metrics and LLM-Judges beyond English](https://mlsanity.com/paper/2503.17039/) (2025)
- [Reward Models are Metrics in a Trench Coat](https://mlsanity.com/paper/2510.03231/) (2025)
- [CNTLS: A Benchmark Dataset for Abstractive or Extractive Chinese Timeline Summarization](https://mlsanity.com/paper/2105.14201/) (2021)
- [A Multi-Document Coverage Reward for RELAXed Multi-Document Summarization](https://mlsanity.com/paper/2203.02894/) (2022)

## Abstract (arXiv)

Abstractive text summarization aims to shorten long text documents into a human readable form that contains the most important facts from the original document. However, the level of actual abstraction as measured by novel phrases that do not appear in the source document remains low in existing approaches. We propose two techniques to improve the level of abstraction of generated summaries. First, we decompose the decoder into a contextual network that retrieves relevant parts of the source document, and a pretrained language model that incorporates prior knowledge about language generation. Second, we propose a novelty metric that is optimized directly through policy learning to encourage the generation of novel phrases. Our model achieves results comparable to state-of-the-art models, as determined by ROUGE scores and human evaluations, while achieving a significantly higher level of abstraction as measured by n-gram overlap with the source document.