# RetrievalSum: A Retrieval Enhanced Framework for Abstractive Summarization

arXiv: https://arxiv.org/abs/2109.07943 · HTML: https://mlsanity.com/paper/2109.07943/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 16 Sep 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://mlsanity.com/paper/1611.04230/) (2016)
- [Extractive Summarization as Text Matching](https://mlsanity.com/paper/2004.08795/) (2020)
- [GSum: A General Framework for Guided Neural Abstractive Summarization](https://mlsanity.com/paper/2010.08014/) (2020)
- [BillSum: A Corpus for Automatic Summarization of US Legislation](https://mlsanity.com/paper/1910.00523/) (2019)
- [Abstractive Summarization of Reddit Posts with Multi-level Memory Networks](https://mlsanity.com/paper/1811.00783/) (2018)
- [Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward](https://mlsanity.com/paper/2005.01159/) (2020)
- [Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization](https://mlsanity.com/paper/1805.05271/) (2018)
- [Searching for Effective Neural Extractive Summarization: What Works and What's Next](https://mlsanity.com/paper/1907.03491/) (2019)
- [Controllable Generation from Pre-trained Language Models via Inverse Prompting](https://mlsanity.com/paper/2103.10685/) (2021)
- [Abstractive Summarization with Combination of Pre-trained Sequence-to-Sequence and Saliency Models](https://mlsanity.com/paper/2003.13028/) (2020)
- [Enhancing Scientific Papers Summarization with Citation Graph](https://mlsanity.com/paper/2104.03057/) (2021)
- [How to Write Summaries with Patterns? Learning towards Abstractive Summarization through Prototype Editing](https://mlsanity.com/paper/1909.08837/) (2019)
- [Contextualized Rewriting for Text Summarization](https://mlsanity.com/paper/2102.00385/) (2021)

## Built on by (most cited first, 19 of 19)

- [Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data](https://mlsanity.com/paper/2203.08773/) (2022)
- [PRCA: Fitting Black-Box Large Language Models for Retrieval Question Answering via Pluggable Reward-Driven Contextual Adapter](https://mlsanity.com/paper/2310.18347/) (2023)
- [On the Evaluation of Machine-Generated Reports](https://mlsanity.com/paper/2405.00982/) (2024)
- [COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization](https://mlsanity.com/paper/2209.14569/) (2022)
- [Unsupervised Multi-Granularity Summarization](https://mlsanity.com/paper/2201.12502/) (2022)
- [On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2502.14759/) (2025)
- [Entity-driven Fact-aware Abstractive Summarization of Biomedical Literature](https://mlsanity.com/paper/2203.15959/) (2022)
- [Meeting Summarization: A Survey of the State of the Art](https://mlsanity.com/paper/2212.08206/) (2022)
- [MUG: A General Meeting Understanding and Generation Benchmark](https://mlsanity.com/paper/2303.13939/) (2023)
- [Improving the Factual Accuracy of Abstractive Clinical Text Summarization using Multi-Objective Optimization](https://mlsanity.com/paper/2204.00797/) (2022)
- [RELexED: Retrieval-Enhanced Legal Summarization with Exemplar Diversity](https://mlsanity.com/paper/2501.14113/) (2025)
- [Magic Mushroom: A Customizable Benchmark for Fine-grained Analysis of Retrieval Noise Erosion in RAG Systems](https://mlsanity.com/paper/2506.03901/) (2025)
- [Document-Level Abstractive Summarization](https://mlsanity.com/paper/2212.03013/) (2022)
- [Controlled Randomness Improves the Performance of Transformer Models](https://mlsanity.com/paper/2310.13526/) (2023)
- [Text Fact Transfer](https://mlsanity.com/paper/2310.14486/) (2023)
- [Non-Parametric Memory Guidance for Multi-Document Summarization](https://mlsanity.com/paper/2311.10760/) (2023)
- [Converging Dimensions: Information Extraction and Summarization through Multisource, Multimodal, and Multilingual Fusion](https://mlsanity.com/paper/2406.13715/) (2024)
- [Towards Optimizing a Retrieval Augmented Generation using Large Language Model on Academic Data](https://mlsanity.com/paper/2411.08438/) (2024)
- [A Unified Retrieval Framework with Document Ranking and EDU Filtering for Multi-document Summarization](https://mlsanity.com/paper/2504.16711/) (2025)

## Abstract (arXiv)

Existing summarization systems mostly generate summaries purely relying on the content of the source document. However, even for humans, we usually need some references or exemplars to help us fully understand the source document and write summaries in a particular format. But how to find the high-quality exemplars and incorporate them into summarization systems is still challenging and worth exploring. In this paper, we propose RetrievalSum, a novel retrieval enhanced abstractive summarization framework consisting of a dense Retriever and a Summarizer. At first, several closely related exemplars are retrieved as supplementary input to help the generation model understand the text more comprehensively. Furthermore, retrieved exemplars can also play a role in guiding the model to capture the writing style of a specific corpus. We validate our method on a wide range of summarization datasets across multiple domains and two backbone models: BERT and BART. Results show that our framework obtains significant improvement by 1.38~4.66 in ROUGE-1 score when compared with the powerful pre-trained models, and achieve new state-of-the-art on BillSum. Human evaluation demonstrates that our retrieval enhanced model can better capture the domain-specific writing style.