# Attend to the Right Context: A Plug-and-Play Module for Content-Controllable Summarization

arXiv: https://arxiv.org/abs/2212.10819 · HTML: https://mlsanity.com/paper/2212.10819/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 21 Dec 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [GeDi: Generative Discriminator Guided Sequence Generation](https://mlsanity.com/paper/2009.06367/) (2020)
- [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021)
- [GSum: A General Framework for Guided Neural Abstractive Summarization](https://mlsanity.com/paper/2010.08014/) (2020)
- [CTRLsum: Towards Generic Controllable Text Summarization](https://mlsanity.com/paper/2012.04281/) (2020)
- [Planning with Learned Entity Prompts for Abstractive Summarization](https://mlsanity.com/paper/2104.07606/) (2021)
- [Sentence Centrality Revisited for Unsupervised Summarization](https://mlsanity.com/paper/1906.03508/) (2019)
- [Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2211.07634/) (2022)
- [Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization](https://mlsanity.com/paper/2208.09770/) (2022)
- [NEWTS: A Corpus for News Topic-Focused Summarization](https://mlsanity.com/paper/2205.15661/) (2022)
- [EntSUM: A Data Set for Entity-Centric Summarization](https://mlsanity.com/paper/2204.02213/) (2022)
- [Inference Time Style Control for Summarization](https://mlsanity.com/paper/2104.01724/) (2021)
- [Relaxed Attention for Transformer Models](https://mlsanity.com/paper/2209.09735/) (2022)

## Built on by (most cited first, 2 of 2)

- [Diversity-Aware Coherence Loss for Improving Neural Topic Models](https://mlsanity.com/paper/2305.16199/) (2023)
- [AspirinSum: an Aspect-based utility-preserved de-identification Summarization framework](https://mlsanity.com/paper/2406.13947/) (2024)

## Abstract (arXiv)

Content-Controllable Summarization generates summaries focused on the given controlling signals. Due to the lack of large-scale training corpora for the task, we propose a plug-and-play module RelAttn to adapt any general summarizers to the content-controllable summarization task. RelAttn first identifies the relevant content in the source documents, and then makes the model attend to the right context by directly steering the attention weight. We further apply an unsupervised online adaptive parameter searching algorithm to determine the degree of control in the zero-shot setting, while such parameters are learned in the few-shot setting. By applying the module to three backbone summarization models, experiments show that our method effectively improves all the summarizers, and outperforms the prefix-based method and a widely used plug-and-play model in both zero- and few-shot settings. Tellingly, more benefit is observed in the scenarios when more control is needed.