# Enhancing Coherence of Extractive Summarization with Multitask Learning

arXiv: https://arxiv.org/abs/2305.12851 · HTML: https://mlsanity.com/paper/2305.12851/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 35 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Fine-tune BERT for Extractive Summarization](https://mlsanity.com/paper/1903.10318/) (2019)
- [Neural Summarization by Extracting Sentences and Words](https://mlsanity.com/paper/1603.07252/) (2016)
- [Heterogeneous Graph Neural Networks for Extractive Document Summarization](https://mlsanity.com/paper/2004.12393/) (2020)
- [Convolutional Neural Network Architectures for Matching Natural Language Sentences](https://mlsanity.com/paper/1503.03244/) (2015)
- [Learning to Extract Coherent Summary via Deep Reinforcement Learning](https://mlsanity.com/paper/1804.07036/) (2018)
- [Extractive Summarization via ChatGPT for Faithful Summary Generation](https://mlsanity.com/paper/2304.04193/) (2023)
- [Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints](https://mlsanity.com/paper/1603.08887/) (2016)
- [A Unified Neural Coherence Model](https://mlsanity.com/paper/1909.00349/) (2019)
- [MemSum: Extractive Summarization of Long Documents Using Multi-Step Episodic Markov Decision Processes](https://mlsanity.com/paper/2107.08929/) (2021)
- [Multiplex Graph Neural Network for Extractive Text Summarization](https://mlsanity.com/paper/2108.12870/) (2021)
- [Can Transformer Models Measure Coherence In Text? Re-Thinking the Shuffle Test](https://mlsanity.com/paper/2107.03448/) (2021)
- [Coherence Modeling of Asynchronous Conversations: A Neural Entity Grid Approach](https://mlsanity.com/paper/1805.02275/) (2018)
- [CNewSum: A Large-scale Chinese News Summarization Dataset with Human-annotated Adequacy and Deducibility Level](https://mlsanity.com/paper/2110.10874/) (2021)

## Built on by (most cited first, 1 of 1)

- [Towards Enhancing Coherence in Extractive Summarization: Dataset and Experiments with LLMs](https://mlsanity.com/paper/2407.04855/) (2024)

## Abstract (arXiv)

This study proposes a multitask learning architecture for extractive summarization with coherence boosting. The architecture contains an extractive summarizer and coherent discriminator module. The coherent discriminator is trained online on the sentence vectors of the augmented textual input, thus improving its general ability of judging whether the input sentences are coherent. Meanwhile, we maximize the coherent scores from the coherent discriminator by updating the parameters of the summarizer. To make the extractive sentences trainable in a differentiable manner, we introduce two strategies, including pre-trained converting model (model-based) and converting matrix (MAT-based) that merge sentence representations. Experiments show that our proposed method significantly improves the proportion of consecutive sentences in the extracted summaries based on their positions in the original article (i.e., automatic sentence-level coherence metric), while the goodness in terms of other automatic metrics (i.e., Rouge scores and BertScores) are preserved. Human evaluation also evidences the improvement of coherence and consistency of the extracted summaries given by our method.