# Preference-based Interactive Multi-Document Summarisation

arXiv: https://arxiv.org/abs/1906.02923 · HTML: https://mlsanity.com/paper/1906.02923/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2019.

## Built on by (most cited first, 11 of 11)

- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Impact of Preference Noise on the Alignment Performance of Generative Language Models](https://mlsanity.com/paper/2404.09824/) (2024)
- [Reward Modeling for Mitigating Toxicity in Transformer-based Language Models](https://mlsanity.com/paper/2202.09662/) (2022)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [Advances in Preference-based Reinforcement Learning: A Review](https://mlsanity.com/paper/2408.11943/) (2024)
- [Hone as You Read: A Practical Type of Interactive Summarization](https://mlsanity.com/paper/2105.02923/) (2021)
- [Interactive Editing for Text Summarization](https://mlsanity.com/paper/2306.03067/) (2023)
- [SummHelper: Collaborative Human-Computer Summarization](https://mlsanity.com/paper/2308.08363/) (2023)
- [SPEC: Summary Preference Decomposition for Low-Resource Abstractive Summarization](https://mlsanity.com/paper/2303.14011/) (2023)
- [Towards Abstractive Timeline Summarisation using Preference-based Reinforcement Learning](https://mlsanity.com/paper/2211.07596/) (2022)
- [TCR-GPT: Integrating Autoregressive Model and Reinforcement Learning for T-Cell Receptor Repertoires Generation](https://mlsanity.com/paper/2408.01156/) (2024)

## Abstract (arXiv)

Interactive NLP is a promising paradigm to close the gap between automatic NLP systems and the human upper bound. Preference-based interactive learning has been successfully applied, but the existing methods require several thousand interaction rounds even in simulations with perfect user feedback. In this paper, we study preference-based interactive summarisation. To reduce the number of interaction rounds, we propose the Active Preference-based ReInforcement Learning (APRIL) framework. APRIL uses Active Learning to query the user, Preference Learning to learn a summary ranking function from the preferences, and neural Reinforcement Learning to efficiently search for the (near-)optimal summary. Our results show that users can easily provide reliable preferences over summaries and that APRIL outperforms the state-of-the-art preference-based interactive method in both simulation and real-user experiments.