# Actor-Critic based Training Framework for Abstractive Summarization

arXiv: https://arxiv.org/abs/1803.11070 · HTML: https://mlsanity.com/paper/1803.11070/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2018.

## Built on by (most cited first, 6 of 6)

- [Abstractive Text Summarization: State of the Art, Challenges, and Improvements](https://mlsanity.com/paper/2409.02413/) (2024)
- [A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization](https://mlsanity.com/paper/2305.08503/) (2023)
- [Reward Models are Metrics in a Trench Coat](https://mlsanity.com/paper/2510.03231/) (2025)
- [PoBRL: Optimizing Multi-Document Summarization by Blending Reinforcement Learning Policies](https://mlsanity.com/paper/2105.08244/) (2021)
- [Abstractive Text Summarization Using the BRIO Training Paradigm](https://mlsanity.com/paper/2305.13696/) (2023)
- [Reinforcement Learning without Human Feedback for Last Mile Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2408.16753/) (2024)

## Abstract (arXiv)

We present a training framework for neural abstractive summarization based on actor-critic approaches from reinforcement learning. In the traditional neural network based methods, the objective is only to maximize the likelihood of the predicted summaries, no other assessment constraints are considered, which may generate low-quality summaries or even incorrect sentences. To alleviate this problem, we employ an actor-critic framework to enhance the training procedure. For the actor, we employ the typical attention based sequence-to-sequence (seq2seq) framework as the policy network for summary generation. For the critic, we combine the maximum likelihood estimator with a well designed global summary quality estimator which is a neural network based binary classifier aiming to make the generated summaries indistinguishable from the human-written ones. Policy gradient method is used to conduct the parameter learning. An alternating training strategy is proposed to conduct the joint training of the actor and critic models. Extensive experiments on some benchmark datasets in different languages show that our framework achieves improvements over the state-of-the-art methods.