# Inference Time Style Control for Summarization

arXiv: https://arxiv.org/abs/2104.01724 · HTML: https://mlsanity.com/paper/2104.01724/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 5 Apr 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [A Neural Attention Model for Abstractive Sentence Summarization](https://mlsanity.com/paper/1509.00685/) (2015)
- [GeDi: Generative Discriminator Guided Sequence Generation](https://mlsanity.com/paper/2009.06367/) (2020)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training](https://mlsanity.com/paper/2001.04063/) (2020)
- [Controllable Abstractive Summarization](https://mlsanity.com/paper/1711.05217/) (2017)
- [Unsupervised Text Style Transfer using Language Models as Discriminators](https://mlsanity.com/paper/1805.11749/) (2018)
- [A Probabilistic Formulation of Unsupervised Text Style Transfer](https://mlsanity.com/paper/2002.03912/) (2020)
- [Polite Dialogue Generation Without Parallel Data](https://mlsanity.com/paper/1805.03162/) (2018)
- [Hooks in the Headline: Learning to Generate Headlines with Controlled Styles](https://mlsanity.com/paper/2004.01980/) (2020)
- [Sentence-Level Content Planning and Style Specification for Neural Text Generation](https://mlsanity.com/paper/1909.00734/) (2019)
- [Semi-supervised Text Style Transfer: Cross Projection in Latent Space](https://mlsanity.com/paper/1909.11493/) (2019)
- [SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation](https://mlsanity.com/paper/1804.04093/) (2018)
- [Accelerated Reinforcement Learning for Sentence Generation by Vocabulary Prediction](https://mlsanity.com/paper/1809.01694/) (2018)

## Built on by (most cited first, 13 of 13)

- [POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection](https://mlsanity.com/paper/2205.00619/) (2022)
- [Aspect-Controllable Opinion Summarization](https://mlsanity.com/paper/2109.03171/) (2021)
- [Comparative Opinion Summarization via Collaborative Decoding](https://mlsanity.com/paper/2110.07520/) (2021)
- [Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Survey](https://mlsanity.com/paper/2311.09212/) (2023)
- [MACSum: Controllable Summarization with Mixed Attributes](https://mlsanity.com/paper/2211.05041/) (2022)
- [RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Framework](https://mlsanity.com/paper/2410.19109/) (2024)
- [Massive-scale Decoding for Text Generation using Lattices](https://mlsanity.com/paper/2112.07660/) (2021)
- [Sequentially Controlled Text Generation](https://mlsanity.com/paper/2301.02299/) (2023)
- [Attend to the Right Context: A Plug-and-Play Module for Content-Controllable Summarization](https://mlsanity.com/paper/2212.10819/) (2022)
- [How to Generate Popular Post Headlines on Social Media?](https://mlsanity.com/paper/2309.09949/) (2023)
- [SPEC: Summary Preference Decomposition for Low-Resource Abstractive Summarization](https://mlsanity.com/paper/2303.14011/) (2023)
- [A Survey of Text Style Transfer: Applications and Ethical Implications](https://mlsanity.com/paper/2407.16737/) (2024)
- [PG-CE: A Progressive Generation Dataset with Constraint Enhancement for Controllable Text Generation](https://mlsanity.com/paper/2509.17669/) (2025)

## Abstract (arXiv)

How to generate summaries of different styles without requiring corpora in the target styles, or training separate models? We present two novel methods that can be deployed during summary decoding on any pre-trained Transformer-based summarization model. (1) Decoder state adjustment instantly modifies decoder final states with externally trained style scorers, to iteratively refine the output against a target style. (2) Word unit prediction constrains the word usage to impose strong lexical control during generation. In experiments of summarizing with simplicity control, automatic evaluation and human judges both find our models producing outputs in simpler languages while still informative. We also generate news headlines with various ideological leanings, which can be distinguished by humans with a reasonable probability.