# A General Contextualized Rewriting Framework for Text Summarization

arXiv: https://arxiv.org/abs/2207.05948 · HTML: https://mlsanity.com/paper/2207.05948/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 13 Jul 2022.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [A Neural Attention Model for Abstractive Sentence Summarization](https://mlsanity.com/paper/1509.00685/) (2015)
- [A Deep Reinforced Model for Abstractive Summarization](https://mlsanity.com/paper/1705.04304/) (2017)
- [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://mlsanity.com/paper/1611.04230/) (2016)
- [Extractive Summarization as Text Matching](https://mlsanity.com/paper/2004.08795/) (2020)
- [Bottom-Up Abstractive Summarization](https://mlsanity.com/paper/1808.10792/) (2018)
- [BRIO: Bringing Order to Abstractive Summarization](https://mlsanity.com/paper/2203.16804/) (2022)
- [Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting](https://mlsanity.com/paper/1805.11080/) (2018)
- [Open-Domain Question Answering Goes Conversational via Question Rewriting](https://mlsanity.com/paper/2010.04898/) (2020)
- [SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization](https://mlsanity.com/paper/2106.01890/) (2021)
- [GSum: A General Framework for Guided Neural Abstractive Summarization](https://mlsanity.com/paper/2010.08014/) (2020)
- [Ranking Sentences for Extractive Summarization with Reinforcement Learning](https://mlsanity.com/paper/1802.08636/) (2018)
- [Neural Summarization by Extracting Sentences and Words](https://mlsanity.com/paper/1603.07252/) (2016)
- [Jointly Learning to Align and Translate with Transformer Models](https://mlsanity.com/paper/1909.02074/) (2019)
- [Improving Multi-turn Dialogue Modelling with Utterance ReWriter](https://mlsanity.com/paper/1906.07004/) (2019)
- [A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss](https://mlsanity.com/paper/1805.06266/) (2018)
- [Learning to Paraphrase for Question Answering](https://mlsanity.com/paper/1708.06022/) (2017)
- [Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints](https://mlsanity.com/paper/1603.08887/) (2016)
- [Summary Level Training of Sentence Rewriting for Abstractive Summarization](https://mlsanity.com/paper/1909.08752/) (2019)
- [Abstractive Summarization with Combination of Pre-trained Sequence-to-Sequence and Saliency Models](https://mlsanity.com/paper/2003.13028/) (2020)
- [Contextualized Rewriting for Text Summarization](https://mlsanity.com/paper/2102.00385/) (2021)

## Built on by (most cited first, 3 of 3)

- [Target-Side Augmentation for Document-Level Machine Translation](https://mlsanity.com/paper/2305.04505/) (2023)
- [GEMINI: Controlling the Sentence-level Writing Style for Abstractive Text Summarization](https://mlsanity.com/paper/2304.03548/) (2023)
- [A Structured Literature Review on Traditional Approaches in Current Natural Language Processing](https://mlsanity.com/paper/2505.12970/) (2025)

## Abstract (arXiv)

The rewriting method for text summarization combines extractive and abstractive approaches, improving the conciseness and readability of extractive summaries using an abstractive model. Exiting rewriting systems take each extractive sentence as the only input, which is relatively focused but can lose necessary background knowledge and discourse context. In this paper, we investigate contextualized rewriting, which consumes the entire document and considers the summary context. We formalize contextualized rewriting as a seq2seq with group-tag alignments, introducing group-tag as a solution to model the alignments, identifying extractive sentences through content-based addressing. Results show that our approach significantly outperforms non-contextualized rewriting systems without requiring reinforcement learning, achieving strong improvements on ROUGE scores upon multiple extractors.