# Towards Abstractive Timeline Summarisation using Preference-based Reinforcement Learning

arXiv: https://arxiv.org/abs/2211.07596 · HTML: https://mlsanity.com/paper/2211.07596/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 14 Nov 2022.

## Builds on (most cited first, 15 of 15)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [A Deep Reinforced Model for Abstractive Summarization](https://mlsanity.com/paper/1705.04304/) (2017)
- [Better Rewards Yield Better Summaries: Learning to Summarise Without References](https://mlsanity.com/paper/1909.01214/) (2019)
- [Better Summarization Evaluation with Word Embeddings for ROUGE](https://mlsanity.com/paper/1508.06034/) (2015)
- [Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning](https://mlsanity.com/paper/1805.10627/) (2018)
- [Examining the State-of-the-Art in News Timeline Summarization](https://mlsanity.com/paper/2005.10107/) (2020)
- [Reinforcement Learning for Abstractive Question Summarization with Question-aware Semantic Rewards](https://mlsanity.com/paper/2107.00176/) (2021)
- [A Temporally Sensitive Submodularity Framework for Timeline Summarization](https://mlsanity.com/paper/1810.07949/) (2018)
- [Preference-based Interactive Multi-Document Summarisation](https://mlsanity.com/paper/1906.02923/) (2019)
- [Revisiting the Centroid-based Method: A Strong Baseline for Multi-Document Summarization](https://mlsanity.com/paper/1708.07690/) (2017)
- [Scalable Bayesian Preference Learning for Crowds](https://mlsanity.com/paper/1912.01987/) (2019)

## Abstract (arXiv)

This paper introduces a novel pipeline for summarising timelines of events reported by multiple news sources. Transformer-based models for abstractive summarisation generate coherent and concise summaries of long documents but can fail to outperform established extractive methods on specialised tasks such as timeline summarisation (TLS). While extractive summaries are more faithful to their sources, they may be less readable and contain redundant or unnecessary information. This paper proposes a preference-based reinforcement learning (PBRL) method for adapting pretrained abstractive summarisers to TLS, which can overcome the drawbacks of extractive timeline summaries. We define a compound reward function that learns from keywords of interest and pairwise preference labels, which we use to fine-tune a pretrained abstractive summariser via offline reinforcement learning. We carry out both automated and human evaluation on three datasets, finding that our method outperforms a comparable extractive TLS method on two of the three benchmark datasets, and participants prefer our method's summaries to those of both the extractive TLS method and the pretrained abstractive model. The method does not require expensive reference summaries and needs only a small number of preferences to align the generated summaries with human preferences.